-- Logs begin at Thu 2022-06-02 18:17:19 UTC, end at Thu 2022-06-02 20:14:10 UTC. -- Jun 02 18:58:51.025587 dv-cinder-830901-8-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 02 18:58:51.034173 dv-cinder-830901-8-iscsi devstack@keystone.service[80481]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 02 18:58:51.036810 dv-cinder-830901-8-iscsi devstack@keystone.service[80481]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jun 2 18:58:51 2022] *** Jun 02 18:58:51.036810 dv-cinder-830901-8-iscsi devstack@keystone.service[80481]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 02 18:58:51.036810 dv-cinder-830901-8-iscsi devstack@keystone.service[80481]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 02 18:58:51.036810 dv-cinder-830901-8-iscsi devstack@keystone.service[80481]: nodename: dv-cinder-830901-8-iscsi Jun 02 18:58:51.036810 dv-cinder-830901-8-iscsi devstack@keystone.service[80481]: machine: x86_64 Jun 02 18:58:51.036810 dv-cinder-830901-8-iscsi devstack@keystone.service[80481]: clock source: unix Jun 02 18:58:51.036810 dv-cinder-830901-8-iscsi devstack@keystone.service[80481]: pcre jit disabled Jun 02 18:58:51.036810 dv-cinder-830901-8-iscsi devstack@keystone.service[80481]: detected number of CPU cores: 4 Jun 02 18:58:51.036810 dv-cinder-830901-8-iscsi devstack@keystone.service[80481]: current working directory: / Jun 02 18:58:51.036810 dv-cinder-830901-8-iscsi devstack@keystone.service[80481]: detected binary path: /usr/bin/uwsgi-core Jun 02 18:58:51.036810 dv-cinder-830901-8-iscsi devstack@keystone.service[80481]: your processes number limit is 47680 Jun 02 18:58:51.036810 dv-cinder-830901-8-iscsi devstack@keystone.service[80481]: your memory page size is 4096 bytes Jun 02 18:58:51.037455 dv-cinder-830901-8-iscsi devstack@keystone.service[80481]: detected max file descriptor number: 2048 Jun 02 18:58:51.037455 dv-cinder-830901-8-iscsi devstack@keystone.service[80481]: lock engine: pthread robust mutexes Jun 02 18:58:51.037455 dv-cinder-830901-8-iscsi devstack@keystone.service[80481]: thunder lock: enabled Jun 02 18:58:51.037455 dv-cinder-830901-8-iscsi devstack@keystone.service[80481]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 02 18:58:51.037455 dv-cinder-830901-8-iscsi devstack@keystone.service[80481]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 02 18:58:51.081188 dv-cinder-830901-8-iscsi devstack@keystone.service[80481]: Python main interpreter initialized at 0x564fbd8009c0 Jun 02 18:58:51.081188 dv-cinder-830901-8-iscsi devstack@keystone.service[80481]: python threads support enabled Jun 02 18:58:51.081188 dv-cinder-830901-8-iscsi devstack@keystone.service[80481]: your server socket listen backlog is limited to 100 connections Jun 02 18:58:51.081188 dv-cinder-830901-8-iscsi devstack@keystone.service[80481]: your mercy for graceful operations on workers is 90 seconds Jun 02 18:58:51.081566 dv-cinder-830901-8-iscsi devstack@keystone.service[80481]: mapped 403077 bytes (393 KB) for 2 cores Jun 02 18:58:51.081643 dv-cinder-830901-8-iscsi devstack@keystone.service[80481]: *** Operational MODE: preforking *** Jun 02 18:58:51.081899 dv-cinder-830901-8-iscsi devstack@keystone.service[80481]: *** uWSGI is running in multiple interpreter mode *** Jun 02 18:58:51.081899 dv-cinder-830901-8-iscsi devstack@keystone.service[80481]: spawned uWSGI master process (pid: 80481) Jun 02 18:58:51.082106 dv-cinder-830901-8-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 02 18:58:51.083515 dv-cinder-830901-8-iscsi devstack@keystone.service[80481]: spawned uWSGI worker 1 (pid: 80491, cores: 1) Jun 02 18:58:51.083620 dv-cinder-830901-8-iscsi devstack@keystone.service[80481]: spawned uWSGI worker 2 (pid: 80492, cores: 1) Jun 02 18:58:51.086698 dv-cinder-830901-8-iscsi devstack@keystone.service[80481]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 02 18:58:56.853874 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 02 18:58:56.853874 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: from cryptography.utils import int_from_bytes Jun 02 18:58:56.854683 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 02 18:58:56.854683 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: from cryptography.utils import int_from_bytes Jun 02 18:58:56.855155 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 02 18:58:56.855155 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: from cryptography.utils import int_from_bytes Jun 02 18:58:56.855281 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 02 18:58:56.855281 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: from cryptography.utils import int_from_bytes Jun 02 18:58:56.904676 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 02 18:58:56.904856 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 02 18:58:56.904932 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 02 18:58:56.905012 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 02 18:58:56.905088 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] command line args: [] {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 02 18:58:56.905164 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] command line args: [] {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 02 18:58:56.905305 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 02 18:58:56.905388 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 02 18:58:56.905463 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ================================================================================ {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 02 18:58:56.905537 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] ================================================================================ {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 02 18:58:56.905723 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] admin_token = **** {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.905812 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] admin_token = **** {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.905967 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] config_dir = [] {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.906048 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] config_dir = [] {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.906155 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.906237 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.906341 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] config_source = [] {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.906426 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] config_source = [] {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.906521 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] debug = True {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.906606 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] debug = True {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.906721 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.908021 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.908021 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.908253 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.908332 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.908446 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.908519 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.908642 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.908722 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.908842 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.908916 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] list_limit = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.909033 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] list_limit = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.909112 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] log_config_append = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.909190 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] log_config_append = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.909303 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.909382 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.909494 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] log_dir = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.909573 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] log_dir = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.909690 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] log_file = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.909774 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] log_file = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.909844 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.909956 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.910044 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.910115 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.910179 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.910255 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.910325 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.910921 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.911043 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.911129 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.911212 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.911335 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.911409 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.911527 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.911607 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.911726 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.911869 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.911949 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.912014 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.912076 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.912138 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.912239 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.912325 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.912396 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.912514 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.912603 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.912679 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.912754 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.912833 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.912904 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.913028 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.33/identity {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.913102 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.33/identity {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.913176 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] publish_errors = False {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.913247 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] publish_errors = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.913360 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.913482 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.913574 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.913657 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.913736 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.913814 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.913976 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.914105 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.914257 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.914338 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.914438 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] standard_threads = False {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.914521 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] standard_threads = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.914666 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.914777 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.914932 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.915019 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.915092 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.915166 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.915271 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] use_journal = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.915350 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] use_journal = False {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.915447 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] use_json = False {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.915527 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] use_json = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.915657 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] use_stderr = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.915792 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] use_stderr = False {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.915792 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] use_syslog = False {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.916043 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] use_syslog = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.916177 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.916307 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 02 18:58:56.916436 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.916565 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.916696 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.916837 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.916962 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.917093 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.917217 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.917345 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.917476 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.917605 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.917731 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.917858 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.917987 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.918111 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.918244 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.918372 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.918499 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.918627 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.918765 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.918859 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.918943 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.919013 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.919082 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.919157 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.919232 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.919302 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.919450 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.919523 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.919640 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.919752 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.919869 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.919980 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.920099 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.920219 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.920337 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.920449 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.920565 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.920684 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.920802 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.920919 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.921037 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.921148 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.921260 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.921378 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.921490 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.921600 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.921716 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.921827 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.921937 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.922065 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.922179 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.922289 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.922400 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.922509 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.922619 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.922739 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.922884 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.922996 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.923106 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.923215 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.923324 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.923448 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.923561 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.923671 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.923780 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.923890 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.923983 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.924087 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.924181 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.924279 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.924378 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.924471 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.924609 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.924729 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.924859 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.924963 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.925076 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.925183 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.925295 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.925416 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.925520 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.925630 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.925740 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.925856 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.925973 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.926674 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.926823 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.926939 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.927050 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.927152 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.927263 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.927382 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.927496 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.927615 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.927734 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.927846 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.927957 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.928067 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.928186 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.928300 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.928422 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.928525 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.928635 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.928752 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.928876 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.928994 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.929108 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.929209 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.929318 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.929435 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.929555 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.929668 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.929780 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.929906 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.930020 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.930121 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.930240 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.930353 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.930463 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.930583 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.930703 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.930841 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.930965 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.931078 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.931189 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.931313 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.931504 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.931619 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.931744 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.931856 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.931967 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.932078 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.932195 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.932306 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.932425 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.932538 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.932649 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.932760 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.932871 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.932982 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.933104 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.933217 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.933328 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.933439 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.933550 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.933661 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.933780 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.933893 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.934005 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.934115 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.934231 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.934343 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.934466 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.934590 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.934702 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.934828 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.934951 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.935084 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.935192 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.935192 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.935409 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.935520 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.935630 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.935740 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.935850 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.935976 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.936089 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.936089 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.936296 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.936296 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.936503 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.936622 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.936734 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.936844 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.936954 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.936954 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.937162 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.937281 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.937392 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.937503 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.937613 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.937725 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.937838 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.937957 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.938071 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.938182 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.938300 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.938412 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.938523 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.938642 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.938817 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.938933 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.939044 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.939156 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.939266 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.939388 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.939502 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.939613 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.939723 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.939834 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.939943 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.939943 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.940161 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.940272 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.940383 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.940383 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.940595 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.940716 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] auth.external = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.940828 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.940939 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.941050 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] auth.external = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.941050 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.941258 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.941403 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.941524 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.941646 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] auth.password = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.941759 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.941870 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] auth.token = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.941982 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] auth.password = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.942093 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.942204 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] auth.token = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.942330 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.942444 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.942555 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.942666 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.942791 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.942904 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.943027 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.943142 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.943253 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.943365 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.943476 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.943587 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.943707 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] credential.caching = True {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.943820 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.943931 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.944043 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] credential.caching = True {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.944154 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.944265 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.944451 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.944567 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.944681 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.944793 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.944904 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.945016 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.945136 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.945250 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.945370 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.945484 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.945596 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.945721 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.945844 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.945957 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.946081 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.946195 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.946306 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.946419 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.946538 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.946657 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.946824 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.946940 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.947052 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.947173 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.947296 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.947410 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.947610 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] federation.caching = True {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.947727 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.947843 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.947963 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] federation.caching = True {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.948077 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.948195 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.948314 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.948427 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.948540 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.948659 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.948773 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.948885 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.948996 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.949113 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.949227 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.949346 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.949460 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.949573 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.949685 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.949796 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.949908 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.950027 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.950141 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.950261 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.950374 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] identity.caching = True {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.950486 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.950604 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.950725 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] identity.caching = True {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.950856 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.950978 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.951092 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.951204 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.951315 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.951435 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.951549 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.951661 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.951772 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.951883 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.951995 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.952114 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.952228 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.952339 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.952451 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.952561 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.952673 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.952794 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.952907 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.953019 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.953131 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.953248 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.953361 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.953481 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.953594 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.953704 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.953815 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.953933 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.954045 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.954165 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.954279 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.954391 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.954503 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.954615 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.954726 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.954868 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.954983 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.955095 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.955207 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.955317 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.955428 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.955547 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.955650 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.955760 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.955873 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.955983 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.956101 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.956221 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.956334 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.956445 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.956556 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.956668 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.956790 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.956904 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.957016 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.957126 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.957237 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.957348 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.957498 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.957615 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.957748 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.957867 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.957980 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.958090 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.958210 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.958322 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.958433 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.958545 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.958655 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.958801 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.958928 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.959044 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.959155 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.959267 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.959378 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.959490 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.959609 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.959723 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.959835 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.959946 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.960057 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.960168 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.960287 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.960401 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.960512 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.960623 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.960734 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.960844 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.960967 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.961081 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.961193 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.961305 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.961416 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.961533 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.961723 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.961840 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.961952 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.962070 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.962185 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] ldap.user = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.962296 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.962420 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.962534 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.user = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.962534 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.962777 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.962898 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.963010 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.963135 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.963252 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.963388 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.963505 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.963638 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.963754 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.963873 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.963987 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.964099 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.964210 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.964323 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.964434 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.964554 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.964668 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.964780 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.964891 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.965002 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.965113 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.965232 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.965346 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.965458 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.965570 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.965680 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.965792 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.965911 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.966025 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.966137 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.966248 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.966361 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.966472 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.966656 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.966820 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.966937 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.967048 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.967159 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.967271 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.967395 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.967510 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.967622 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.967733 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.967848 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.967959 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.968079 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.968193 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.968304 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.968416 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.968528 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.968639 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.968759 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.968872 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.968983 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.969101 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.969214 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.969333 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.969453 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.969568 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.969680 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.969792 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.969903 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.970014 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.970135 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.970249 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.970360 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.970472 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.970584 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.970702 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.970842 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] resource.caching = True {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.970964 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.971078 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.971190 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] resource.caching = True {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.971302 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.971421 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.971535 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.971648 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.971759 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.971870 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.971981 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.972101 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.972214 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.972326 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.972437 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.972548 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.972661 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.972780 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.972893 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.973499 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.973610 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] role.caching = True {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.973723 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.973835 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] role.driver = sql {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.973954 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] role.caching = True {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.974067 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.974185 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] role.driver = sql {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.974298 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.974409 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.974520 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.974640 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.974817 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.974933 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.975046 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.975158 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.975269 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.975392 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.975507 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.975619 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.975731 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.975843 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.975954 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.976073 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.976187 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.976298 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.976409 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.976520 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.976631 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.976751 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.976865 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.976977 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.977088 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.977200 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.977451 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.977574 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.977688 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.977852 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.977969 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.978086 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.978197 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.978317 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.978431 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.978572 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.978688 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.978859 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.978989 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.979105 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.979218 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.979330 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.979441 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.979579 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.979704 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.979817 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.979927 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.980040 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.980151 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.980261 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.980381 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.980495 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.980606 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.980717 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.980828 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.980965 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.981091 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.981204 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.981316 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.981427 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.981538 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.981649 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.981769 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.981881 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] token.caching = True {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.981992 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.982104 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.982216 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.982327 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.982448 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] token.caching = True {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.982561 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.982673 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.982811 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.982926 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.983060 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.983188 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.983302 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.983413 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.983524 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.983635 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.983746 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.983865 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.983978 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.984090 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.984203 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.984314 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.984425 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.984544 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.984657 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.984769 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.984880 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.985008 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.985120 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.985241 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.985354 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.985466 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.985577 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.985688 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.985844 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.985969 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.986084 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.986195 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] database.connection = **** {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.986342 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.986466 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.986579 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.986691 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] database.connection = **** {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.986905 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.987031 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.987152 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.987266 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.987379 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.987490 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.987601 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.987713 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.987837 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.987951 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.988063 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.988180 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.988292 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.988405 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.988525 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.989226 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.989342 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.989455 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.989567 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.989679 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.989800 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.989914 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.990026 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.990137 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.990249 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.990361 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.990483 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.990597 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.990710 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.990869 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.990982 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.991094 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.991218 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80492) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 02 18:58:56.991334 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.991334 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 02 18:58:56.991544 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 02 18:58:57.021051 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80492) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 02 18:58:57.021275 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 02 18:58:57.021361 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80492) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 02 18:58:57.021504 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80492) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 02 18:58:57.021586 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 02 18:58:57.021731 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 02 18:58:57.030255 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80492) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 02 18:58:57.030346 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 02 18:58:57.059521 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 02 18:58:57.059660 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80492) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 02 18:58:57.059738 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 02 18:58:57.059812 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80492) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 02 18:58:57.115948 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 02 18:58:57.116287 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80492) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 02 18:58:57.165504 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 02 18:58:57.165732 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 02 18:58:57.166118 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80492) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 02 18:58:57.166356 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80492) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 02 18:58:57.233728 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 02 18:58:57.233955 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 02 18:58:57.234062 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80492) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 02 18:58:57.234183 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 02 18:58:57.234268 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80492) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 02 18:58:57.234407 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 02 18:58:57.234489 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80492) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 02 18:58:57.234625 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80492) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 02 18:58:57.234705 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 02 18:58:57.234872 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 02 18:58:57.234955 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80492) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 02 18:58:57.235092 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80492) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 02 18:58:57.258339 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 02 18:58:57.258548 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80492) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 02 18:58:57.258658 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 02 18:58:57.258802 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80492) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 02 18:58:57.258899 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 02 18:58:57.259034 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 02 18:58:57.259114 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80492) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 02 18:58:57.259256 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 02 18:58:57.259338 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80492) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 02 18:58:57.259474 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80492) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 02 18:58:57.259553 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 02 18:58:57.259692 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 02 18:58:57.259780 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80492) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 02 18:58:57.259905 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80492) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 02 18:58:57.271616 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 02 18:58:57.271922 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80492) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 02 18:58:57.284191 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 02 18:58:57.284430 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 02 18:58:57.284661 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 02 18:58:57.284743 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80492) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 02 18:58:57.284887 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 02 18:58:57.284968 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80492) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 02 18:58:57.285124 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 02 18:58:57.285211 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80492) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 02 18:58:57.285333 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 02 18:58:57.285453 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80492) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 02 18:58:57.286130 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80492) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 02 18:58:57.286217 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 02 18:58:57.286355 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80492) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 02 18:58:57.286603 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80492) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 02 18:58:57.303824 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.304039 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 02 18:58:57.304360 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.304443 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.304568 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 02 18:58:57.304648 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 02 18:58:57.304843 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.304922 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.305090 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 02 18:58:57.305177 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.305361 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.305440 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.305616 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.305694 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.305873 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.306131 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.316757 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.317541 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.317816 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.317942 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.318092 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.318199 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.318476 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.318775 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.326265 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 02 18:58:57.326941 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80492) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 02 18:58:57.329608 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 02 18:58:57.329893 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.330181 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.330260 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80492) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 02 18:58:57.330466 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.330559 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.330770 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.330852 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.331084 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.331163 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.331368 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.331450 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.331683 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.331761 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.331956 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.332054 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.332232 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.332329 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.332529 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.332609 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.332849 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.333118 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.359516 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.360396 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.361233 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 02 18:58:57.361516 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.362131 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80492) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 02 18:58:57.362411 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.367455 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 02 18:58:57.367691 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.367974 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.368367 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80492) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 02 18:58:57.368654 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.368939 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.375559 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 02 18:58:57.375844 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.376588 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80492) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 02 18:58:57.376873 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.380406 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 02 18:58:57.380804 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.381090 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.381377 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.381536 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80492) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 02 18:58:57.381659 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.381840 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.381944 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.382123 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.382212 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.382405 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.382492 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.382680 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.382979 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.383251 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.383520 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.401292 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.401571 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.402053 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.402330 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.406273 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 02 18:58:57.406980 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80492) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 02 18:58:57.409977 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.410274 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.411262 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.411581 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.415953 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 02 18:58:57.416545 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80492) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 02 18:58:57.419637 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 02 18:58:57.420153 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80492) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 02 18:58:57.424813 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.425120 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.425292 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.425418 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.425597 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.425691 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.425886 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.425969 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.426777 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.426884 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.427069 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.427350 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.445863 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 02 18:58:57.446151 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.446299 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80492) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 02 18:58:57.446422 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.446588 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.446698 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.446914 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.447204 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.454701 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.455216 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.456470 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.456747 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.456934 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.457210 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.459843 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 02 18:58:57.460128 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.460411 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.460492 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80492) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 02 18:58:57.460690 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.460782 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.460981 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.461061 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.461266 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.461550 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.474977 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 02 18:58:57.475264 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.475547 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.475628 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80492) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 02 18:58:57.475835 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.475929 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.476115 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.476193 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.476392 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.476472 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.476665 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.476745 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.476938 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.477211 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.495582 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 02 18:58:57.497076 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80492) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 02 18:58:57.498884 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 02 18:58:57.500653 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80492) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 02 18:58:57.502471 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.504006 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.504096 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.505645 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.505797 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 02 18:58:57.506076 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.506355 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.507309 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80492) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 02 18:58:57.507588 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.507867 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.513845 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.515390 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.515526 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 02 18:58:57.517013 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80492) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 02 18:58:57.518742 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.519059 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.519349 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.519629 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.520333 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.520622 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.520905 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.521180 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.528849 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 02 18:58:57.529139 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.529526 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.530265 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80492) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 02 18:58:57.530557 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.530968 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.538280 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 02 18:58:57.538558 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.538862 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.539138 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.539410 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.539672 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80492) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 02 18:58:57.539765 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.539950 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.540040 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.540226 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.540308 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.540493 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.540573 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.540762 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.540839 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.541031 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.541108 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.541301 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.541381 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.541574 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.541654 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.541924 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.542011 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.542197 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.542474 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.542758 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.543043 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.543317 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80492) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 02 18:58:57.580977 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x564fbd8009c0 pid: 80491 (default app) Jun 02 18:58:57.581767 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x564fbd8009c0 pid: 80492 (default app) Jun 02 18:58:57.584401 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11daa3f2-482b-46ea-a4bc-2004e9854613 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:58:57.584699 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11daa3f2-482b-46ea-a4bc-2004e9854613 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:58:57.584980 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11daa3f2-482b-46ea-a4bc-2004e9854613 None None] PATH_INFO: `/v3` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:58:57.589300 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1/1] 10.222.0.33 () {50 vars in 791 bytes} [Thu Jun 2 18:58:57 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:06.545742 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0576a78-1044-4e07-baac-4ed80a5a9d59 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:06.546300 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0576a78-1044-4e07-baac-4ed80a5a9d59 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:06.546300 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0576a78-1044-4e07-baac-4ed80a5a9d59 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:06.550837 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1/2] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 18:59:06 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 02 18:59:06.556196 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137102ea-bc2b-4ad7-9dd7-940ecd109c33 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:06.556594 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137102ea-bc2b-4ad7-9dd7-940ecd109c33 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:06.556747 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137102ea-bc2b-4ad7-9dd7-940ecd109c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:06.623244 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: INFO dbcounter [None req-137102ea-bc2b-4ad7-9dd7-940ecd109c33 None None] Registered counter for database keystone Jun 02 18:59:06.644943 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG oslo_db.sqlalchemy.engines [None req-137102ea-bc2b-4ad7-9dd7-940ecd109c33 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80491) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 02 18:59:06.651877 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writer thread running {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 02 18:59:06.935166 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG passlib.handlers.bcrypt [None req-137102ea-bc2b-4ad7-9dd7-940ecd109c33 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80491) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 02 18:59:06.935491 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG passlib.handlers.bcrypt [None req-137102ea-bc2b-4ad7-9dd7-940ecd109c33 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80491) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 02 18:59:06.991466 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-137102ea-bc2b-4ad7-9dd7-940ecd109c33 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 18:59:06.993075 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-137102ea-bc2b-4ad7-9dd7-940ecd109c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 18:59:07.119149 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 2/3] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 18:59:06 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 564 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 02 18:59:07.286725 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0576a78-1044-4e07-baac-4ed80a5a9d59 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:07.354780 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: INFO dbcounter [None req-c0576a78-1044-4e07-baac-4ed80a5a9d59 None None] Registered counter for database keystone Jun 02 18:59:07.376022 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG oslo_db.sqlalchemy.engines [None req-c0576a78-1044-4e07-baac-4ed80a5a9d59 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80492) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 02 18:59:07.382888 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writer thread running {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 02 18:59:07.607388 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0576a78-1044-4e07-baac-4ed80a5a9d59 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:07.611632 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33af90d-1006-4136-9415-c13e7bd1b814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:07.612307 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33af90d-1006-4136-9415-c13e7bd1b814 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:07.612597 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33af90d-1006-4136-9415-c13e7bd1b814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:07.612877 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33af90d-1006-4136-9415-c13e7bd1b814 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:07.621728 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33af90d-1006-4136-9415-c13e7bd1b814 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:07.675174 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d33af90d-1006-4136-9415-c13e7bd1b814 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 02 18:59:07.675571 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33af90d-1006-4136-9415-c13e7bd1b814 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:07.685856 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 2/4] 10.222.0.33 () {58 vars in 1324 bytes} [Thu Jun 2 18:59:07 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 02 18:59:07.691915 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137102ea-bc2b-4ad7-9dd7-940ecd109c33 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:07.715581 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137102ea-bc2b-4ad7-9dd7-940ecd109c33 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:07.720151 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9abd66-f9eb-42f5-8eec-8fcc7a9cbd81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:07.721291 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9abd66-f9eb-42f5-8eec-8fcc7a9cbd81 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:07.721583 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9abd66-f9eb-42f5-8eec-8fcc7a9cbd81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:07.721877 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9abd66-f9eb-42f5-8eec-8fcc7a9cbd81 admin admin] PATH_INFO: `/v3/projects/879ee62507da4a8282b2418099da56e5` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:07.724019 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9abd66-f9eb-42f5-8eec-8fcc7a9cbd81 admin admin] RBAC: Authorizing `identity:get_project(project_id=879ee62507da4a8282b2418099da56e5)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:07.775799 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: WARNING keystone.common.rbac_enforcer.enforcer [None req-9a9abd66-f9eb-42f5-8eec-8fcc7a9cbd81 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 02 18:59:07.776140 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9abd66-f9eb-42f5-8eec-8fcc7a9cbd81 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:07.780920 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 3/5] 10.222.0.33 () {56 vars in 1207 bytes} [Thu Jun 2 18:59:07 2022] GET /identity/v3/projects/879ee62507da4a8282b2418099da56e5 => generated 344 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:10.029371 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16546bbf-2eba-419b-93e3-050e7166bdb3 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:10.029920 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16546bbf-2eba-419b-93e3-050e7166bdb3 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:10.029920 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16546bbf-2eba-419b-93e3-050e7166bdb3 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:10.033986 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 3/6] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 18:59:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 18:59:10.039479 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cfe31c-bf1c-44cf-a427-0aef4e820a6e None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:10.039914 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cfe31c-bf1c-44cf-a427-0aef4e820a6e None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:10.040059 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cfe31c-bf1c-44cf-a427-0aef4e820a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:10.059265 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-08cfe31c-bf1c-44cf-a427-0aef4e820a6e None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 18:59:10.060807 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-08cfe31c-bf1c-44cf-a427-0aef4e820a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 18:59:10.069309 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 4/7] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 18:59:10 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 02 18:59:10.091527 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16546bbf-2eba-419b-93e3-050e7166bdb3 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:10.104277 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16546bbf-2eba-419b-93e3-050e7166bdb3 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:10.108495 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a854f7-89d9-4375-8a66-9cc2c92645a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:10.109131 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a854f7-89d9-4375-8a66-9cc2c92645a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:10.109419 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a854f7-89d9-4375-8a66-9cc2c92645a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:10.109699 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a854f7-89d9-4375-8a66-9cc2c92645a9 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:10.118372 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a854f7-89d9-4375-8a66-9cc2c92645a9 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:10.120200 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a854f7-89d9-4375-8a66-9cc2c92645a9 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:10.129402 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 4/8] 10.222.0.33 () {58 vars in 1324 bytes} [Thu Jun 2 18:59:10 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 02 18:59:10.135056 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cfe31c-bf1c-44cf-a427-0aef4e820a6e None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:10.144079 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cfe31c-bf1c-44cf-a427-0aef4e820a6e None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:10.148537 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7165b562-ae39-4132-9d16-0e8f9e7a35b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:10.149405 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7165b562-ae39-4132-9d16-0e8f9e7a35b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:10.149693 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7165b562-ae39-4132-9d16-0e8f9e7a35b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:10.149971 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7165b562-ae39-4132-9d16-0e8f9e7a35b1 admin admin] PATH_INFO: `/v3/users/789d0dc0871243f79a6f4f6f602c4f21` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:10.151406 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7165b562-ae39-4132-9d16-0e8f9e7a35b1 admin admin] RBAC: Authorizing `identity:get_user(user_id=789d0dc0871243f79a6f4f6f602c4f21)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:10.153253 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7165b562-ae39-4132-9d16-0e8f9e7a35b1 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:10.154834 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 5/9] 10.222.0.33 () {56 vars in 1198 bytes} [Thu Jun 2 18:59:10 2022] GET /identity/v3/users/789d0dc0871243f79a6f4f6f602c4f21 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:12.487216 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dbd095-8ae5-4fcd-8968-905dd25d90b9 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:12.487720 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dbd095-8ae5-4fcd-8968-905dd25d90b9 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:12.487816 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dbd095-8ae5-4fcd-8968-905dd25d90b9 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:12.489665 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 5/10] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 18:59:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 18:59:12.494907 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e404937c-e319-4cca-a5bc-3474050d6f1b None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:12.495319 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e404937c-e319-4cca-a5bc-3474050d6f1b None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:12.495473 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e404937c-e319-4cca-a5bc-3474050d6f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:12.512181 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-e404937c-e319-4cca-a5bc-3474050d6f1b None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 18:59:12.513361 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-e404937c-e319-4cca-a5bc-3474050d6f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 18:59:12.548491 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9abe8e8-8650-4f83-a6af-fb7d0ae12bdc None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:12.548779 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9abe8e8-8650-4f83-a6af-fb7d0ae12bdc None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:12.549063 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9abe8e8-8650-4f83-a6af-fb7d0ae12bdc None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:12.550464 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 6/11] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 18:59:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 02 18:59:12.557193 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfdf1a9-38c4-49db-88dd-b5782e9e2975 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:12.557480 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfdf1a9-38c4-49db-88dd-b5782e9e2975 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:12.557760 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfdf1a9-38c4-49db-88dd-b5782e9e2975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:12.596313 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 6/12] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 18:59:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 18:59:12.597464 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG passlib.handlers.bcrypt [None req-9cfdf1a9-38c4-49db-88dd-b5782e9e2975 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80492) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 02 18:59:12.597901 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG passlib.handlers.bcrypt [None req-9cfdf1a9-38c4-49db-88dd-b5782e9e2975 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80492) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 02 18:59:12.620243 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e404937c-e319-4cca-a5bc-3474050d6f1b None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:12.644322 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:12.646209 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-9cfdf1a9-38c4-49db-88dd-b5782e9e2975 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 18:59:12.647854 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-9cfdf1a9-38c4-49db-88dd-b5782e9e2975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 18:59:12.733835 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e404937c-e319-4cca-a5bc-3474050d6f1b None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:12.750642 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 7/13] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 18:59:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 02 18:59:12.771688 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf9ce01-3138-4a00-b2f9-a6ad9bf7487c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:12.772499 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf9ce01-3138-4a00-b2f9-a6ad9bf7487c None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:12.772786 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf9ce01-3138-4a00-b2f9-a6ad9bf7487c None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:12.773068 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf9ce01-3138-4a00-b2f9-a6ad9bf7487c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:12.773744 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfdf1a9-38c4-49db-88dd-b5782e9e2975 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:12.784729 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf9ce01-3138-4a00-b2f9-a6ad9bf7487c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 18:59:12.796234 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-3cf9ce01-3138-4a00-b2f9-a6ad9bf7487c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 02 18:59:12.796234 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 18:59:12.796234 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 18:59:12.796234 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 18:59:12.796234 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 18:59:12.796234 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 18:59:12.796234 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 18:59:12.796234 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 18:59:12.796234 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 18:59:12.796234 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 18:59:12.796234 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 18:59:12.796234 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 18:59:12.796234 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 18:59:12.796234 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 18:59:12.796234 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 18:59:12.796234 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 18:59:12.797148 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 18:59:12.797148 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 18:59:12.797148 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 18:59:12.797148 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 18:59:12.797148 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 18:59:12.797148 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 18:59:12.797148 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 18:59:12.797148 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 18:59:12.797148 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 18:59:12.797148 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 18:59:12.797148 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 18:59:12.797148 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 18:59:12.797148 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 18:59:12.797148 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 18:59:12.797148 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 18:59:12.797148 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 18:59:12.798234 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 18:59:12.798234 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 18:59:12.798234 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 18:59:12.798234 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 18:59:12.798234 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 18:59:12.798234 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 18:59:12.798234 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 18:59:12.798234 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 18:59:12.798234 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 18:59:12.798234 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 18:59:12.798234 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 02 18:59:12.798234 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 18:59:12.798234 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 7/14] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 2 18:59:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 02 18:59:12.803891 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf9ce01-3138-4a00-b2f9-a6ad9bf7487c None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:12.874539 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfdf1a9-38c4-49db-88dd-b5782e9e2975 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:12.877144 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:12.886478 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:12.904825 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf9ce01-3138-4a00-b2f9-a6ad9bf7487c None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:12.912893 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de14ab50-d312-4bca-8b2d-649d60f63e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:12.913803 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de14ab50-d312-4bca-8b2d-649d60f63e65 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:12.914098 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de14ab50-d312-4bca-8b2d-649d60f63e65 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:12.914385 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de14ab50-d312-4bca-8b2d-649d60f63e65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:12.939704 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cf0335-8f0c-48de-bdad-09ef28808a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:12.940375 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cf0335-8f0c-48de-bdad-09ef28808a5c None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:12.940663 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cf0335-8f0c-48de-bdad-09ef28808a5c None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:12.940951 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cf0335-8f0c-48de-bdad-09ef28808a5c None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:12.941768 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cf0335-8f0c-48de-bdad-09ef28808a5c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 18:59:12.942088 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cf0335-8f0c-48de-bdad-09ef28808a5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:12.943563 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cf0335-8f0c-48de-bdad-09ef28808a5c None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:12.956422 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 8/15] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 2 18:59:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 18:59:12.962415 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cf0335-8f0c-48de-bdad-09ef28808a5c None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:12.981604 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de14ab50-d312-4bca-8b2d-649d60f63e65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:12.983168 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de14ab50-d312-4bca-8b2d-649d60f63e65 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:13.062254 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cf0335-8f0c-48de-bdad-09ef28808a5c None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:13.084736 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 8/16] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 2 18:59:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 02 18:59:13.090323 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de14ab50-d312-4bca-8b2d-649d60f63e65 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:13.093125 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:13.097761 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98abe9b0-e943-4d92-ab46-9ff84fa4a9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:13.099239 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98abe9b0-e943-4d92-ab46-9ff84fa4a9ef None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:13.099528 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98abe9b0-e943-4d92-ab46-9ff84fa4a9ef None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:13.099806 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98abe9b0-e943-4d92-ab46-9ff84fa4a9ef None admin] PATH_INFO: `/v3/users/789d0dc0871243f79a6f4f6f602c4f21` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:13.101039 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98abe9b0-e943-4d92-ab46-9ff84fa4a9ef None admin] RBAC: Authorizing `identity:get_user(user_id=789d0dc0871243f79a6f4f6f602c4f21)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:13.102503 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98abe9b0-e943-4d92-ab46-9ff84fa4a9ef None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:13.104347 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 9/17] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 2 18:59:12 2022] GET /identity/v3/users/789d0dc0871243f79a6f4f6f602c4f21 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 18:59:13.109473 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98abe9b0-e943-4d92-ab46-9ff84fa4a9ef None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:13.124554 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:13.192187 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de14ab50-d312-4bca-8b2d-649d60f63e65 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:13.213943 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98abe9b0-e943-4d92-ab46-9ff84fa4a9ef None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:13.227953 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5006d1-9d88-48d9-8886-dcc9a7c2cba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:13.228765 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5006d1-9d88-48d9-8886-dcc9a7c2cba4 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:13.229104 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5006d1-9d88-48d9-8886-dcc9a7c2cba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:13.229343 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5006d1-9d88-48d9-8886-dcc9a7c2cba4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:13.254471 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5006d1-9d88-48d9-8886-dcc9a7c2cba4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:13.256989 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5006d1-9d88-48d9-8886-dcc9a7c2cba4 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:13.262776 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ed74dd-cc58-4bd6-a13a-d7aa68abfb3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:13.263616 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ed74dd-cc58-4bd6-a13a-d7aa68abfb3f None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:13.263905 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ed74dd-cc58-4bd6-a13a-d7aa68abfb3f None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:13.264181 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ed74dd-cc58-4bd6-a13a-d7aa68abfb3f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:13.265430 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ed74dd-cc58-4bd6-a13a-d7aa68abfb3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:13.266985 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ed74dd-cc58-4bd6-a13a-d7aa68abfb3f None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:13.268692 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 10/18] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 18:59:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 18:59:13.274709 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ed74dd-cc58-4bd6-a13a-d7aa68abfb3f None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:13.282471 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-eb5006d1-9d88-48d9-8886-dcc9a7c2cba4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 18:59:13.282471 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 18:59:13.282471 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 18:59:13.282471 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 18:59:13.282471 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 18:59:13.282471 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 18:59:13.282471 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 18:59:13.282471 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 18:59:13.282471 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 18:59:13.282471 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 18:59:13.282471 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 18:59:13.282471 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 18:59:13.282471 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 18:59:13.282471 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 18:59:13.282471 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 18:59:13.283671 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 18:59:13.283671 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 18:59:13.283671 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 18:59:13.283671 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 18:59:13.283671 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 18:59:13.283671 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 18:59:13.283671 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 18:59:13.283671 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 18:59:13.283671 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 18:59:13.283671 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 18:59:13.283671 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 18:59:13.283671 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 18:59:13.283671 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 18:59:13.283671 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 18:59:13.283671 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 18:59:13.283671 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 18:59:13.283671 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 18:59:13.284649 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 18:59:13.284649 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 18:59:13.284649 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 18:59:13.284649 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 18:59:13.284649 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 9/19] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 18:59:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 02 18:59:13.291985 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5006d1-9d88-48d9-8886-dcc9a7c2cba4 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:13.356272 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:13.360564 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:13.383150 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ed74dd-cc58-4bd6-a13a-d7aa68abfb3f None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:13.413523 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5006d1-9d88-48d9-8886-dcc9a7c2cba4 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:13.418812 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eafe220-173d-4134-bedc-42ae85a58ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:13.419479 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eafe220-173d-4134-bedc-42ae85a58ff8 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:13.419767 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eafe220-173d-4134-bedc-42ae85a58ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:13.420044 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eafe220-173d-4134-bedc-42ae85a58ff8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:13.420884 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eafe220-173d-4134-bedc-42ae85a58ff8 None admin] RBAC: Adding query filter params (role.id=f13fdd0847924014abf1e95594195e78, scope.domain.id=default, user.id=789d0dc0871243f79a6f4f6f602c4f21) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 18:59:13.421197 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eafe220-173d-4134-bedc-42ae85a58ff8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:13.422798 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eafe220-173d-4134-bedc-42ae85a58ff8 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:13.431811 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 11/20] 10.222.0.33 () {56 vars in 1322 bytes} [Thu Jun 2 18:59:13 2022] GET /identity/v3/role_assignments?user.id=789d0dc0871243f79a6f4f6f602c4f21&scope.domain.id=default&role.id=f13fdd0847924014abf1e95594195e78 => generated 233 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 18:59:13.468276 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b4b0ae-da91-41b4-9e9b-92134df948c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:13.469477 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b4b0ae-da91-41b4-9e9b-92134df948c0 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:13.469981 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b4b0ae-da91-41b4-9e9b-92134df948c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:13.470497 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b4b0ae-da91-41b4-9e9b-92134df948c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:13.471939 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b4b0ae-da91-41b4-9e9b-92134df948c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 18:59:13.472478 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b4b0ae-da91-41b4-9e9b-92134df948c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:13.475493 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b4b0ae-da91-41b4-9e9b-92134df948c0 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:13.512559 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 10/21] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 18:59:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 18:59:16.516538 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf85b7-01f3-45e0-a0d1-1f832da495e7 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:16.517249 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf85b7-01f3-45e0-a0d1-1f832da495e7 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:16.517249 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf85b7-01f3-45e0-a0d1-1f832da495e7 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:16.518890 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 12/22] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 18:59:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 18:59:16.538400 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e73dae-500f-43ac-a7ff-f6685fd1722d None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:16.539101 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e73dae-500f-43ac-a7ff-f6685fd1722d None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:16.539410 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e73dae-500f-43ac-a7ff-f6685fd1722d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:16.580607 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-17e73dae-500f-43ac-a7ff-f6685fd1722d None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 18:59:16.582557 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-17e73dae-500f-43ac-a7ff-f6685fd1722d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 18:59:16.741130 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 11/23] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 18:59:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 18:59:16.778238 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cf85b7-01f3-45e0-a0d1-1f832da495e7 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:17.017487 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cf85b7-01f3-45e0-a0d1-1f832da495e7 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:17.095688 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f8110b-c2d9-40ed-b7ce-d10a2c99f9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:17.096826 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f8110b-c2d9-40ed-b7ce-d10a2c99f9de None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:17.097287 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f8110b-c2d9-40ed-b7ce-d10a2c99f9de None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:17.097731 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f8110b-c2d9-40ed-b7ce-d10a2c99f9de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:17.121442 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f8110b-c2d9-40ed-b7ce-d10a2c99f9de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:17.124475 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f8110b-c2d9-40ed-b7ce-d10a2c99f9de None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:17.153812 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:17.189713 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-53f8110b-c2d9-40ed-b7ce-d10a2c99f9de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 18:59:17.189713 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 18:59:17.189713 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 18:59:17.189713 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 18:59:17.189713 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 18:59:17.189713 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 18:59:17.189713 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 18:59:17.189713 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 18:59:17.189713 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 18:59:17.189713 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 18:59:17.189713 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 18:59:17.189713 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 18:59:17.189713 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 18:59:17.189713 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 18:59:17.189713 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 18:59:17.190590 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 18:59:17.190590 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 18:59:17.190590 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 18:59:17.190590 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 18:59:17.190590 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 18:59:17.190590 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 18:59:17.190590 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 18:59:17.190590 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 18:59:17.190590 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 18:59:17.190590 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 18:59:17.190590 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 18:59:17.190590 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 18:59:17.190590 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 18:59:17.190590 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 18:59:17.190590 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 18:59:17.190590 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 18:59:17.190590 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 18:59:17.191442 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 18:59:17.191442 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 18:59:17.191442 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 18:59:17.191442 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 18:59:17.192044 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 13/24] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 18:59:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 415 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 18:59:17.201363 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e73dae-500f-43ac-a7ff-f6685fd1722d None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:17.323127 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:17.449367 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e73dae-500f-43ac-a7ff-f6685fd1722d None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:17.524606 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71504f56-7f57-4628-90c8-01e13b33d049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:17.525747 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71504f56-7f57-4628-90c8-01e13b33d049 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:17.526244 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71504f56-7f57-4628-90c8-01e13b33d049 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:17.526727 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71504f56-7f57-4628-90c8-01e13b33d049 None admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:17.528171 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71504f56-7f57-4628-90c8-01e13b33d049 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 18:59:17.528717 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71504f56-7f57-4628-90c8-01e13b33d049 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:17.531231 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71504f56-7f57-4628-90c8-01e13b33d049 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:17.560595 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 12/25] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 18:59:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:17.569405 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f8110b-c2d9-40ed-b7ce-d10a2c99f9de None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:17.771157 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f8110b-c2d9-40ed-b7ce-d10a2c99f9de None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:17.851806 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08085cd7-2e92-43e3-82d3-59d7d610b362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:17.853422 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08085cd7-2e92-43e3-82d3-59d7d610b362 None admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:17.854190 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08085cd7-2e92-43e3-82d3-59d7d610b362 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:17.855000 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08085cd7-2e92-43e3-82d3-59d7d610b362 None admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:17.856835 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08085cd7-2e92-43e3-82d3-59d7d610b362 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:17.859918 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08085cd7-2e92-43e3-82d3-59d7d610b362 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:17.931125 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a36e3a-5332-408a-bd84-b444dd715b05 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:17.931607 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a36e3a-5332-408a-bd84-b444dd715b05 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:17.932092 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a36e3a-5332-408a-bd84-b444dd715b05 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:17.935030 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 13/26] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 18:59:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 18:59:17.944024 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc199a88-4cee-487b-adee-3e91e91757d1 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:17.944518 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc199a88-4cee-487b-adee-3e91e91757d1 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:17.945015 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc199a88-4cee-487b-adee-3e91e91757d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:17.976067 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-dc199a88-4cee-487b-adee-3e91e91757d1 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 18:59:17.979465 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-dc199a88-4cee-487b-adee-3e91e91757d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 18:59:18.088426 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 14/27] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 18:59:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 02 18:59:18.164723 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc199a88-4cee-487b-adee-3e91e91757d1 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:18.236990 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:18.350890 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc199a88-4cee-487b-adee-3e91e91757d1 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:18.385366 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 14/28] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Jun 2 18:59:17 2022] POST /identity/v3/projects => generated 301 bytes in 818 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 18:59:18.413652 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3276396-87a3-4feb-a3cf-c7d94eb6cf18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:18.414480 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3276396-87a3-4feb-a3cf-c7d94eb6cf18 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:18.414813 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3276396-87a3-4feb-a3cf-c7d94eb6cf18 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:18.415098 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3276396-87a3-4feb-a3cf-c7d94eb6cf18 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:18.437098 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3276396-87a3-4feb-a3cf-c7d94eb6cf18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 18:59:18.449336 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346c2d80-da45-4345-ad05-ccf8eb48e821 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:18.449815 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346c2d80-da45-4345-ad05-ccf8eb48e821 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:18.450322 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346c2d80-da45-4345-ad05-ccf8eb48e821 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:18.453452 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 15/29] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 18:59:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 18:59:18.455302 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-b3276396-87a3-4feb-a3cf-c7d94eb6cf18 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 02 18:59:18.455302 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 18:59:18.455302 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 18:59:18.455302 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 18:59:18.455302 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 18:59:18.455302 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 18:59:18.455302 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 18:59:18.455302 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 18:59:18.455302 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 18:59:18.455302 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 18:59:18.455302 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 18:59:18.455302 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 18:59:18.455302 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 18:59:18.455302 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 18:59:18.455302 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 18:59:18.455302 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 18:59:18.456828 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 18:59:18.456828 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 18:59:18.456828 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 18:59:18.456828 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 18:59:18.456828 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 18:59:18.456828 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 18:59:18.456828 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 18:59:18.456828 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 18:59:18.456828 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 18:59:18.456828 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 18:59:18.456828 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 18:59:18.456828 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 18:59:18.456828 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 18:59:18.456828 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 18:59:18.456828 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 18:59:18.456828 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 18:59:18.458951 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 18:59:18.458951 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 18:59:18.458951 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 18:59:18.458951 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 18:59:18.458951 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 18:59:18.458951 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 18:59:18.458951 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 18:59:18.458951 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 18:59:18.458951 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 18:59:18.458951 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 18:59:18.458951 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 02 18:59:18.458951 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 18:59:18.458951 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 15/30] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 2 18:59:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 02 18:59:18.465415 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c615f-efcb-4408-b0a6-c8c202a3fa03 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:18.466337 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c615f-efcb-4408-b0a6-c8c202a3fa03 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:18.466891 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c615f-efcb-4408-b0a6-c8c202a3fa03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:18.472220 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3276396-87a3-4feb-a3cf-c7d94eb6cf18 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:18.540637 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-0e2c615f-efcb-4408-b0a6-c8c202a3fa03 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 18:59:18.543590 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-0e2c615f-efcb-4408-b0a6-c8c202a3fa03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 18:59:18.648678 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:18.655894 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3276396-87a3-4feb-a3cf-c7d94eb6cf18 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:18.710789 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:18.733078 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c81bc3-8dfd-4ddf-94d9-5edf4adfff9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:18.733819 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c81bc3-8dfd-4ddf-94d9-5edf4adfff9a None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:18.734108 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c81bc3-8dfd-4ddf-94d9-5edf4adfff9a None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:18.734390 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c81bc3-8dfd-4ddf-94d9-5edf4adfff9a None admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:18.735203 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c81bc3-8dfd-4ddf-94d9-5edf4adfff9a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 18:59:18.735517 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c81bc3-8dfd-4ddf-94d9-5edf4adfff9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:18.736947 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c81bc3-8dfd-4ddf-94d9-5edf4adfff9a None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:18.763450 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 16/31] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 2 18:59:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:18.772479 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 16/32] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 18:59:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 18:59:18.777868 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c81bc3-8dfd-4ddf-94d9-5edf4adfff9a None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:18.823277 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2c615f-efcb-4408-b0a6-c8c202a3fa03 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:18.915528 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c81bc3-8dfd-4ddf-94d9-5edf4adfff9a None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:18.994127 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c924207-d9ad-480a-8bf0-6309cbadae69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:18.995050 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c924207-d9ad-480a-8bf0-6309cbadae69 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:18.995332 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c924207-d9ad-480a-8bf0-6309cbadae69 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:18.995611 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c924207-d9ad-480a-8bf0-6309cbadae69 None admin] PATH_INFO: `/v3/users/789d0dc0871243f79a6f4f6f602c4f21` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:18.997347 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c924207-d9ad-480a-8bf0-6309cbadae69 None admin] RBAC: Authorizing `identity:get_user(user_id=789d0dc0871243f79a6f4f6f602c4f21)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:18.998846 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c924207-d9ad-480a-8bf0-6309cbadae69 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:19.001601 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 17/33] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 2 18:59:18 2022] GET /identity/v3/users/789d0dc0871243f79a6f4f6f602c4f21 => generated 247 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 18:59:19.028568 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c924207-d9ad-480a-8bf0-6309cbadae69 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:19.066300 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2c615f-efcb-4408-b0a6-c8c202a3fa03 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:19.067637 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:19.186494 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3acc405-0aee-4255-98cc-401653f1a70b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:19.187560 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3acc405-0aee-4255-98cc-401653f1a70b None admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:19.188024 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3acc405-0aee-4255-98cc-401653f1a70b None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:19.188489 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3acc405-0aee-4255-98cc-401653f1a70b None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:19.189508 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3acc405-0aee-4255-98cc-401653f1a70b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:19.191320 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3acc405-0aee-4255-98cc-401653f1a70b None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:19.202662 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c924207-d9ad-480a-8bf0-6309cbadae69 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:19.251515 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4acf083-ad5e-456d-972d-9d347f0a74bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:19.252951 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4acf083-ad5e-456d-972d-9d347f0a74bd None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:19.253457 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4acf083-ad5e-456d-972d-9d347f0a74bd None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:19.254024 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4acf083-ad5e-456d-972d-9d347f0a74bd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:19.256455 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4acf083-ad5e-456d-972d-9d347f0a74bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:19.259448 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4acf083-ad5e-456d-972d-9d347f0a74bd None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:19.262784 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 18/34] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 18:59:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 18:59:19.272134 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4acf083-ad5e-456d-972d-9d347f0a74bd None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:19.307352 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 17/35] 10.222.0.33 () {60 vars in 1131 bytes} [Thu Jun 2 18:59:18 2022] POST /identity/v3/roles => generated 219 bytes in 485 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 18:59:19.309264 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624aeeb8-1589-4b05-98a2-c9707cadf3ca None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:19.309757 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624aeeb8-1589-4b05-98a2-c9707cadf3ca None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:19.310228 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624aeeb8-1589-4b05-98a2-c9707cadf3ca None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:19.312136 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 18/36] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 18:59:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 18:59:19.327313 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2080abc2-1e06-464e-a4f5-acdd14bbb60e None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:19.327969 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2080abc2-1e06-464e-a4f5-acdd14bbb60e None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:19.328457 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2080abc2-1e06-464e-a4f5-acdd14bbb60e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:19.369166 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-2080abc2-1e06-464e-a4f5-acdd14bbb60e None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 18:59:19.372616 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-2080abc2-1e06-464e-a4f5-acdd14bbb60e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 18:59:19.445530 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:19.459128 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:19.471335 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4acf083-ad5e-456d-972d-9d347f0a74bd None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:19.524035 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 19/37] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 18:59:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 02 18:59:19.526901 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d76e3b3-7f7f-479f-8c8b-acdab6e5d321 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:19.527476 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d76e3b3-7f7f-479f-8c8b-acdab6e5d321 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:19.528025 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d76e3b3-7f7f-479f-8c8b-acdab6e5d321 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:19.531499 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 20/38] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 18:59:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 18:59:19.537390 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82142069-ef3a-479d-bc58-b99f06beaf70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:19.539138 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82142069-ef3a-479d-bc58-b99f06beaf70 None admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:19.539687 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82142069-ef3a-479d-bc58-b99f06beaf70 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:19.540241 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82142069-ef3a-479d-bc58-b99f06beaf70 None admin] PATH_INFO: `/v3/domains/default/users/789d0dc0871243f79a6f4f6f602c4f21/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:19.543265 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52eb50a-1720-49ca-a071-80e1cbf75f6c None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:19.543866 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52eb50a-1720-49ca-a071-80e1cbf75f6c None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:19.544326 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52eb50a-1720-49ca-a071-80e1cbf75f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:19.544703 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82142069-ef3a-479d-bc58-b99f06beaf70 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=789d0dc0871243f79a6f4f6f602c4f21, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:19.547783 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82142069-ef3a-479d-bc58-b99f06beaf70 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:19.572711 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-d52eb50a-1720-49ca-a071-80e1cbf75f6c None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 18:59:19.575055 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-d52eb50a-1720-49ca-a071-80e1cbf75f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 18:59:19.699587 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 21/39] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 18:59:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 02 18:59:19.701218 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52eb50a-1720-49ca-a071-80e1cbf75f6c None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:19.860143 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 19/40] 10.222.0.33 () {58 vars in 1361 bytes} [Thu Jun 2 18:59:19 2022] PUT /identity/v3/domains/default/users/789d0dc0871243f79a6f4f6f602c4f21/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 02 18:59:19.861828 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82142069-ef3a-479d-bc58-b99f06beaf70 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:19.899214 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52eb50a-1720-49ca-a071-80e1cbf75f6c None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:19.903292 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:19.968654 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4be26f7-5f00-4bbb-991f-4ff3eff4f5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:19.969825 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be26f7-5f00-4bbb-991f-4ff3eff4f5db None admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:19.970324 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be26f7-5f00-4bbb-991f-4ff3eff4f5db None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:19.970841 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be26f7-5f00-4bbb-991f-4ff3eff4f5db None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:19.972272 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4be26f7-5f00-4bbb-991f-4ff3eff4f5db None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:19.975609 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4be26f7-5f00-4bbb-991f-4ff3eff4f5db None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:20.003397 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82142069-ef3a-479d-bc58-b99f06beaf70 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:20.045516 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf7d54b-18cd-4721-9eb1-472d5dbdcfb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:20.046259 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf7d54b-18cd-4721-9eb1-472d5dbdcfb1 None admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:20.046584 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf7d54b-18cd-4721-9eb1-472d5dbdcfb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:20.046953 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf7d54b-18cd-4721-9eb1-472d5dbdcfb1 None admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:20.047898 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf7d54b-18cd-4721-9eb1-472d5dbdcfb1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:20.049601 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf7d54b-18cd-4721-9eb1-472d5dbdcfb1 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:20.080750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 22/41] 10.222.0.33 () {60 vars in 1131 bytes} [Thu Jun 2 18:59:19 2022] POST /identity/v3/roles => generated 223 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 18:59:20.082262 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819faa37-52cf-4a5e-baa3-ac603720f6b8 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:20.088057 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819faa37-52cf-4a5e-baa3-ac603720f6b8 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:20.088057 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819faa37-52cf-4a5e-baa3-ac603720f6b8 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:20.088057 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 23/42] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 18:59:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 18:59:20.091547 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d279abb-4a4a-48c3-8ba8-f2b164e7ba24 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:20.091846 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d279abb-4a4a-48c3-8ba8-f2b164e7ba24 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:20.092146 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d279abb-4a4a-48c3-8ba8-f2b164e7ba24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:20.111488 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-9d279abb-4a4a-48c3-8ba8-f2b164e7ba24 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 18:59:20.120732 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-9d279abb-4a4a-48c3-8ba8-f2b164e7ba24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 18:59:20.165659 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 20/43] 10.222.0.33 () {60 vars in 1131 bytes} [Thu Jun 2 18:59:19 2022] POST /identity/v3/roles => generated 225 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 18:59:20.219819 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 24/44] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 18:59:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 02 18:59:20.246442 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf7d54b-18cd-4721-9eb1-472d5dbdcfb1 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:20.267890 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:20.393480 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf7d54b-18cd-4721-9eb1-472d5dbdcfb1 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:20.458475 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b78b4d6-8b9c-49a8-9264-bcc35652ac8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:20.460168 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b78b4d6-8b9c-49a8-9264-bcc35652ac8f None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:20.460816 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b78b4d6-8b9c-49a8-9264-bcc35652ac8f None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:20.461548 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b78b4d6-8b9c-49a8-9264-bcc35652ac8f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:20.463650 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b78b4d6-8b9c-49a8-9264-bcc35652ac8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:20.465244 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b78b4d6-8b9c-49a8-9264-bcc35652ac8f None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:20.467384 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 21/45] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 18:59:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 18:59:20.473385 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d279abb-4a4a-48c3-8ba8-f2b164e7ba24 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:20.601536 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:20.614699 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d279abb-4a4a-48c3-8ba8-f2b164e7ba24 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:20.654088 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62c0943-73bd-4bdb-afb3-f19a40cad80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:20.654806 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62c0943-73bd-4bdb-afb3-f19a40cad80f None admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:20.655101 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62c0943-73bd-4bdb-afb3-f19a40cad80f None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:20.655385 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62c0943-73bd-4bdb-afb3-f19a40cad80f None admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:20.656139 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62c0943-73bd-4bdb-afb3-f19a40cad80f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:20.657663 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62c0943-73bd-4bdb-afb3-f19a40cad80f None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:20.739199 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 25/46] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Jun 2 18:59:20 2022] POST /identity/v3/projects => generated 312 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 18:59:22.471101 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abb2e54-209c-477f-ae85-7a130a357fb0 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:22.471648 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abb2e54-209c-477f-ae85-7a130a357fb0 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:22.471800 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abb2e54-209c-477f-ae85-7a130a357fb0 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:22.473068 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 22/47] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 18:59:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 18:59:22.479375 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a7fa37-67e8-4bb7-b292-3906d0c14461 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:22.479857 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a7fa37-67e8-4bb7-b292-3906d0c14461 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:22.480045 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a7fa37-67e8-4bb7-b292-3906d0c14461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:22.498176 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-f2a7fa37-67e8-4bb7-b292-3906d0c14461 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 18:59:22.499412 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-f2a7fa37-67e8-4bb7-b292-3906d0c14461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 18:59:22.570662 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 26/48] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 18:59:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 18:59:22.594364 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abb2e54-209c-477f-ae85-7a130a357fb0 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:22.684452 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:22.704373 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abb2e54-209c-477f-ae85-7a130a357fb0 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:22.743437 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e198052c-7dc3-4732-ab25-9c76c3351038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:22.744292 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e198052c-7dc3-4732-ab25-9c76c3351038 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:22.744659 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e198052c-7dc3-4732-ab25-9c76c3351038 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:22.745025 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e198052c-7dc3-4732-ab25-9c76c3351038 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:22.756716 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e198052c-7dc3-4732-ab25-9c76c3351038 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 18:59:22.764559 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-e198052c-7dc3-4732-ab25-9c76c3351038 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 02 18:59:22.764559 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 18:59:22.764559 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 18:59:22.764559 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 18:59:22.764559 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 18:59:22.764559 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 18:59:22.764559 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 18:59:22.764559 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 18:59:22.764559 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 18:59:22.764559 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 18:59:22.764559 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 18:59:22.764559 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 18:59:22.764559 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 18:59:22.764559 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 18:59:22.764559 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 18:59:22.764559 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 18:59:22.771597 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 18:59:22.771597 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 18:59:22.771597 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 18:59:22.771597 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 18:59:22.771597 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 18:59:22.771597 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 18:59:22.771597 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 18:59:22.771597 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 18:59:22.771597 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 18:59:22.771597 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 18:59:22.771597 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 18:59:22.771597 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 18:59:22.771597 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 18:59:22.771597 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 18:59:22.771597 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 18:59:22.771597 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 18:59:22.772510 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 18:59:22.772510 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 18:59:22.772510 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 18:59:22.772510 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 18:59:22.772510 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 18:59:22.772510 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 18:59:22.772510 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 18:59:22.772510 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 18:59:22.772510 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 18:59:22.772510 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 18:59:22.772510 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 02 18:59:22.772510 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 18:59:22.772510 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 23/49] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 2 18:59:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 18:59:22.773139 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a7fa37-67e8-4bb7-b292-3906d0c14461 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:22.851894 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:22.886253 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a7fa37-67e8-4bb7-b292-3906d0c14461 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:22.923174 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6575cec7-d28e-472f-ad8a-f5e0f54c9b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:22.923860 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575cec7-d28e-472f-ad8a-f5e0f54c9b92 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:22.924152 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575cec7-d28e-472f-ad8a-f5e0f54c9b92 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:22.924440 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575cec7-d28e-472f-ad8a-f5e0f54c9b92 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:22.925212 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6575cec7-d28e-472f-ad8a-f5e0f54c9b92 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 18:59:22.925527 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6575cec7-d28e-472f-ad8a-f5e0f54c9b92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:22.927596 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6575cec7-d28e-472f-ad8a-f5e0f54c9b92 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:22.936891 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 27/50] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 2 18:59:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:22.945046 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e198052c-7dc3-4732-ab25-9c76c3351038 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:23.059749 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e198052c-7dc3-4732-ab25-9c76c3351038 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:23.096536 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df6452a-d8c0-43fd-88dc-cfe0b0080683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:23.097564 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df6452a-d8c0-43fd-88dc-cfe0b0080683 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:23.097920 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df6452a-d8c0-43fd-88dc-cfe0b0080683 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:23.098271 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df6452a-d8c0-43fd-88dc-cfe0b0080683 None admin] PATH_INFO: `/v3/users/789d0dc0871243f79a6f4f6f602c4f21` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:23.100176 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df6452a-d8c0-43fd-88dc-cfe0b0080683 None admin] RBAC: Authorizing `identity:get_user(user_id=789d0dc0871243f79a6f4f6f602c4f21)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:23.101759 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df6452a-d8c0-43fd-88dc-cfe0b0080683 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:23.103798 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 24/51] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 2 18:59:22 2022] GET /identity/v3/users/789d0dc0871243f79a6f4f6f602c4f21 => generated 247 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:23.110155 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6575cec7-d28e-472f-ad8a-f5e0f54c9b92 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:23.173573 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffda1dba-596b-48c0-a60e-8e43b8d3bd37 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:23.173862 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffda1dba-596b-48c0-a60e-8e43b8d3bd37 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:23.174146 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffda1dba-596b-48c0-a60e-8e43b8d3bd37 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:23.175574 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 25/52] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 18:59:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 18:59:23.181536 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c179c920-9b66-4a23-bc4c-50886ff0a178 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:23.181825 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c179c920-9b66-4a23-bc4c-50886ff0a178 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:23.182104 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c179c920-9b66-4a23-bc4c-50886ff0a178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:23.199240 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-c179c920-9b66-4a23-bc4c-50886ff0a178 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 18:59:23.200792 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-c179c920-9b66-4a23-bc4c-50886ff0a178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 18:59:23.203906 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:23.274830 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 26/53] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 18:59:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 02 18:59:23.290439 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6575cec7-d28e-472f-ad8a-f5e0f54c9b92 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:23.300535 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c179c920-9b66-4a23-bc4c-50886ff0a178 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:23.331748 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:23.337705 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756289ad-d4da-4634-a6f4-5c81e7b853e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:23.338509 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756289ad-d4da-4634-a6f4-5c81e7b853e1 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:23.338827 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756289ad-d4da-4634-a6f4-5c81e7b853e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:23.339113 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756289ad-d4da-4634-a6f4-5c81e7b853e1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:23.340361 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756289ad-d4da-4634-a6f4-5c81e7b853e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:23.341904 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756289ad-d4da-4634-a6f4-5c81e7b853e1 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:23.347114 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 28/54] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 18:59:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:23.351021 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756289ad-d4da-4634-a6f4-5c81e7b853e1 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:23.422474 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c179c920-9b66-4a23-bc4c-50886ff0a178 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:23.468418 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaa9ec5-1a34-49cc-be5c-9684e297aae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:23.469339 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaa9ec5-1a34-49cc-be5c-9684e297aae5 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:23.469728 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaa9ec5-1a34-49cc-be5c-9684e297aae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:23.470156 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaa9ec5-1a34-49cc-be5c-9684e297aae5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:23.471566 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaa9ec5-1a34-49cc-be5c-9684e297aae5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:23.473529 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaa9ec5-1a34-49cc-be5c-9684e297aae5 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:23.476148 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 27/55] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 18:59:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 18:59:23.480818 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756289ad-d4da-4634-a6f4-5c81e7b853e1 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:23.482918 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaa9ec5-1a34-49cc-be5c-9684e297aae5 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:23.500096 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:23.517962 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc213bc8-9018-4340-95ac-f4f63baf82c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:23.518708 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc213bc8-9018-4340-95ac-f4f63baf82c7 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:23.519138 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc213bc8-9018-4340-95ac-f4f63baf82c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:23.519488 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc213bc8-9018-4340-95ac-f4f63baf82c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:23.520421 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc213bc8-9018-4340-95ac-f4f63baf82c7 None admin] RBAC: Adding query filter params (role.id=f13fdd0847924014abf1e95594195e78, scope.domain.id=default, user.id=789d0dc0871243f79a6f4f6f602c4f21) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 18:59:23.520808 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc213bc8-9018-4340-95ac-f4f63baf82c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:23.522350 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc213bc8-9018-4340-95ac-f4f63baf82c7 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:23.528726 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 29/56] 10.222.0.33 () {56 vars in 1322 bytes} [Thu Jun 2 18:59:23 2022] GET /identity/v3/role_assignments?user.id=789d0dc0871243f79a6f4f6f602c4f21&scope.domain.id=default&role.id=f13fdd0847924014abf1e95594195e78 => generated 530 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 18:59:23.671194 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaa9ec5-1a34-49cc-be5c-9684e297aae5 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:23.737742 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b882d0c7-b6f0-453f-823e-a882d703cd01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:23.738895 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b882d0c7-b6f0-453f-823e-a882d703cd01 None admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:23.739388 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b882d0c7-b6f0-453f-823e-a882d703cd01 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:23.739866 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b882d0c7-b6f0-453f-823e-a882d703cd01 None admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:23.741123 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b882d0c7-b6f0-453f-823e-a882d703cd01 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:23.743804 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b882d0c7-b6f0-453f-823e-a882d703cd01 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:23.836916 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 28/57] 10.222.0.33 () {60 vars in 1140 bytes} [Thu Jun 2 18:59:23 2022] POST /identity/v3/projects => generated 298 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 02 18:59:26.236309 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b48dc66-fa19-4369-8036-a5d3eb4ec157 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:26.236829 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b48dc66-fa19-4369-8036-a5d3eb4ec157 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:26.236957 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b48dc66-fa19-4369-8036-a5d3eb4ec157 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:26.238419 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 30/58] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 18:59:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 18:59:26.245178 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c33006-578c-4435-97cd-65b5184df6bc None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:26.245717 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c33006-578c-4435-97cd-65b5184df6bc None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:26.245830 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c33006-578c-4435-97cd-65b5184df6bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:26.264267 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-d9c33006-578c-4435-97cd-65b5184df6bc None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 18:59:26.265571 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-d9c33006-578c-4435-97cd-65b5184df6bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 18:59:26.326352 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:26.338273 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 29/59] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 18:59:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 18:59:26.365095 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b48dc66-fa19-4369-8036-a5d3eb4ec157 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:26.433487 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:26.473789 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b48dc66-fa19-4369-8036-a5d3eb4ec157 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:26.511406 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91979cb0-899d-45e9-b27d-3852306da56b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:26.512222 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91979cb0-899d-45e9-b27d-3852306da56b None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:26.512515 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91979cb0-899d-45e9-b27d-3852306da56b None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:26.512799 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91979cb0-899d-45e9-b27d-3852306da56b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:26.514037 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91979cb0-899d-45e9-b27d-3852306da56b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:26.515531 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91979cb0-899d-45e9-b27d-3852306da56b None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:26.517205 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 31/60] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 18:59:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:26.523272 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c33006-578c-4435-97cd-65b5184df6bc None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:26.624708 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c33006-578c-4435-97cd-65b5184df6bc None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:26.663324 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4999aa88-58a2-47d0-9532-f8493eabc538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:26.664005 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4999aa88-58a2-47d0-9532-f8493eabc538 None admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:26.664294 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4999aa88-58a2-47d0-9532-f8493eabc538 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:26.664601 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4999aa88-58a2-47d0-9532-f8493eabc538 None admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:26.665389 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4999aa88-58a2-47d0-9532-f8493eabc538 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:26.666873 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4999aa88-58a2-47d0-9532-f8493eabc538 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:26.764301 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 30/61] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 2 18:59:26 2022] POST /identity/v3/users => generated 275 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 02 18:59:31.133340 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ebe5d-d91e-4cf4-90b6-0c5775e21559 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:31.134386 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ebe5d-d91e-4cf4-90b6-0c5775e21559 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:31.134909 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ebe5d-d91e-4cf4-90b6-0c5775e21559 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:31.136528 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 32/62] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 18:59:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 18:59:31.143795 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8171e66b-c7a6-49a1-bed4-1f95ff40b087 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:31.144320 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8171e66b-c7a6-49a1-bed4-1f95ff40b087 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:31.144788 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8171e66b-c7a6-49a1-bed4-1f95ff40b087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:31.184081 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-8171e66b-c7a6-49a1-bed4-1f95ff40b087 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 18:59:31.186120 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-8171e66b-c7a6-49a1-bed4-1f95ff40b087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 18:59:31.292786 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:31.353771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 31/63] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 18:59:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 18:59:31.430036 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5ebe5d-d91e-4cf4-90b6-0c5775e21559 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:31.532230 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab19df50-49d4-42e1-a5f8-e0881243eb92 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:31.532735 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab19df50-49d4-42e1-a5f8-e0881243eb92 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:31.533231 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab19df50-49d4-42e1-a5f8-e0881243eb92 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:31.536061 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 32/64] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 18:59:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 18:59:31.549528 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c938d275-b6ac-4b2d-8c2b-7eb782b284c2 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:31.550043 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c938d275-b6ac-4b2d-8c2b-7eb782b284c2 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:31.550548 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c938d275-b6ac-4b2d-8c2b-7eb782b284c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:31.596666 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-c938d275-b6ac-4b2d-8c2b-7eb782b284c2 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 18:59:31.598775 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-c938d275-b6ac-4b2d-8c2b-7eb782b284c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 18:59:31.603794 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5ebe5d-d91e-4cf4-90b6-0c5775e21559 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:31.657662 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:31.666795 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b98af5-99e3-4f62-a329-931d39173588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:31.668069 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b98af5-99e3-4f62-a329-931d39173588 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:31.668530 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b98af5-99e3-4f62-a329-931d39173588 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:31.675173 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b98af5-99e3-4f62-a329-931d39173588 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:31.689517 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b98af5-99e3-4f62-a329-931d39173588 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 18:59:31.704850 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-d0b98af5-99e3-4f62-a329-931d39173588 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 02 18:59:31.704850 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 18:59:31.704850 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 18:59:31.704850 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 18:59:31.704850 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 18:59:31.704850 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 18:59:31.704850 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 18:59:31.704850 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 18:59:31.704850 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 18:59:31.704850 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 18:59:31.704850 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 18:59:31.704850 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 18:59:31.704850 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 18:59:31.704850 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 18:59:31.704850 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 18:59:31.704850 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 18:59:31.706720 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 18:59:31.706720 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 18:59:31.706720 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 18:59:31.706720 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 18:59:31.706720 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 18:59:31.706720 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 18:59:31.706720 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 18:59:31.706720 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 18:59:31.706720 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 18:59:31.706720 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 18:59:31.706720 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 18:59:31.706720 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 18:59:31.706720 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 18:59:31.706720 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 18:59:31.706720 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 18:59:31.706720 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 18:59:31.708236 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 18:59:31.708236 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 18:59:31.708236 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 18:59:31.708236 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 18:59:31.708236 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 18:59:31.708236 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 18:59:31.708236 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 18:59:31.708236 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 18:59:31.708236 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 18:59:31.708236 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 18:59:31.708236 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 02 18:59:31.708236 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 18:59:31.709882 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 33/65] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 2 18:59:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 18:59:31.714912 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab40fd-9417-4645-a81f-0e18dea65b0a None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:31.714912 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab40fd-9417-4645-a81f-0e18dea65b0a None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:31.714912 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab40fd-9417-4645-a81f-0e18dea65b0a None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:31.720651 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 34/66] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 18:59:31 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 18:59:31.722149 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ab40fd-9417-4645-a81f-0e18dea65b0a None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:31.784342 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 33/67] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 18:59:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 02 18:59:31.789097 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5700e94-f6a1-447b-8d48-a0df3a8ba823 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:31.789719 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5700e94-f6a1-447b-8d48-a0df3a8ba823 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:31.790308 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5700e94-f6a1-447b-8d48-a0df3a8ba823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:31.820268 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-a5700e94-f6a1-447b-8d48-a0df3a8ba823 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 18:59:31.822397 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-a5700e94-f6a1-447b-8d48-a0df3a8ba823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 18:59:31.869348 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ab40fd-9417-4645-a81f-0e18dea65b0a None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:31.901771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 34/68] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 18:59:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 18:59:31.911190 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e3ac9-2287-4e17-a55f-c7937bb6c612 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:31.911190 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e3ac9-2287-4e17-a55f-c7937bb6c612 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:31.911190 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e3ac9-2287-4e17-a55f-c7937bb6c612 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:31.911190 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 35/69] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 18:59:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 18:59:31.911190 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f167e3-548a-4341-93f7-67ad4749ec40 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:31.911190 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f167e3-548a-4341-93f7-67ad4749ec40 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:31.912028 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f167e3-548a-4341-93f7-67ad4749ec40 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:31.912028 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 36/70] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 18:59:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 18:59:31.912141 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bc9946-0ae1-453e-85b7-899092fea038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:31.912223 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f167e3-548a-4341-93f7-67ad4749ec40 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:31.912296 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc9946-0ae1-453e-85b7-899092fea038 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:31.912296 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc9946-0ae1-453e-85b7-899092fea038 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:31.912417 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc9946-0ae1-453e-85b7-899092fea038 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:31.914244 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bc9946-0ae1-453e-85b7-899092fea038 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 18:59:31.914986 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bc9946-0ae1-453e-85b7-899092fea038 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:31.920137 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bc9946-0ae1-453e-85b7-899092fea038 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:31.934621 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 35/71] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 2 18:59:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:31.936006 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aa755d-86a9-4505-bc62-8f3aeb2f75a1 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:31.936300 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aa755d-86a9-4505-bc62-8f3aeb2f75a1 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:31.936581 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aa755d-86a9-4505-bc62-8f3aeb2f75a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:31.954110 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-00aa755d-86a9-4505-bc62-8f3aeb2f75a1 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 18:59:31.955283 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-00aa755d-86a9-4505-bc62-8f3aeb2f75a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 18:59:31.961040 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:32.020717 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:32.042256 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f167e3-548a-4341-93f7-67ad4749ec40 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:32.042882 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 36/72] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 18:59:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 18:59:32.044277 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c762fe-f24e-49bf-b5b2-0011092ee4f7 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:32.044642 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c762fe-f24e-49bf-b5b2-0011092ee4f7 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:32.044924 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c762fe-f24e-49bf-b5b2-0011092ee4f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:32.063814 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-e5c762fe-f24e-49bf-b5b2-0011092ee4f7 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 18:59:32.065058 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-e5c762fe-f24e-49bf-b5b2-0011092ee4f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 18:59:32.096773 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63be542c-0ca7-4a37-bb00-76ab832d0aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:32.097585 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63be542c-0ca7-4a37-bb00-76ab832d0aa7 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:32.097865 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63be542c-0ca7-4a37-bb00-76ab832d0aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:32.098141 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63be542c-0ca7-4a37-bb00-76ab832d0aa7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:32.109745 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63be542c-0ca7-4a37-bb00-76ab832d0aa7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 18:59:32.118796 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-63be542c-0ca7-4a37-bb00-76ab832d0aa7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 02 18:59:32.118796 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 18:59:32.118796 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 18:59:32.118796 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 18:59:32.118796 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 18:59:32.118796 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 18:59:32.118796 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 18:59:32.118796 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 18:59:32.118796 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 18:59:32.118796 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 18:59:32.118796 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 18:59:32.118796 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 18:59:32.118796 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 18:59:32.118796 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 18:59:32.118796 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 18:59:32.118796 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 18:59:32.120108 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 18:59:32.120108 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 18:59:32.120108 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 18:59:32.120108 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 18:59:32.120108 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 18:59:32.120108 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 18:59:32.120108 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 18:59:32.120108 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 18:59:32.120108 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 18:59:32.120108 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 18:59:32.120108 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 18:59:32.120108 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 18:59:32.120108 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 18:59:32.120108 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 18:59:32.120108 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 18:59:32.120108 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 18:59:32.122469 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 18:59:32.122469 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 18:59:32.122469 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 18:59:32.122469 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 18:59:32.122469 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 18:59:32.122469 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 18:59:32.122469 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 18:59:32.122469 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 18:59:32.122469 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 18:59:32.122469 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 18:59:32.122469 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 02 18:59:32.122469 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 18:59:32.122469 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 37/73] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 2 18:59:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 18:59:32.125535 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63be542c-0ca7-4a37-bb00-76ab832d0aa7 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:32.144532 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 37/74] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 18:59:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 18:59:32.145409 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c762fe-f24e-49bf-b5b2-0011092ee4f7 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:32.262126 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:32.267167 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c762fe-f24e-49bf-b5b2-0011092ee4f7 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:32.268823 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63be542c-0ca7-4a37-bb00-76ab832d0aa7 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:32.283651 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:32.306349 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa10ad52-c5f5-4933-a191-8b819c9910d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:32.307204 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa10ad52-c5f5-4933-a191-8b819c9910d8 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:32.307590 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa10ad52-c5f5-4933-a191-8b819c9910d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:32.307883 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa10ad52-c5f5-4933-a191-8b819c9910d8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:32.315169 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9142c9-6138-47d2-b1cf-8619a7673016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:32.316878 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9142c9-6138-47d2-b1cf-8619a7673016 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:32.317755 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9142c9-6138-47d2-b1cf-8619a7673016 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:32.318261 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9142c9-6138-47d2-b1cf-8619a7673016 None admin] PATH_INFO: `/v3/users/789d0dc0871243f79a6f4f6f602c4f21` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:32.319361 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa10ad52-c5f5-4933-a191-8b819c9910d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 18:59:32.320433 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9142c9-6138-47d2-b1cf-8619a7673016 None admin] RBAC: Authorizing `identity:get_user(user_id=789d0dc0871243f79a6f4f6f602c4f21)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:32.323226 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9142c9-6138-47d2-b1cf-8619a7673016 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:32.332261 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 38/75] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 2 18:59:32 2022] GET /identity/v3/users/789d0dc0871243f79a6f4f6f602c4f21 => generated 247 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:32.332498 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-aa10ad52-c5f5-4933-a191-8b819c9910d8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 02 18:59:32.332498 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 18:59:32.332498 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 18:59:32.332498 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 18:59:32.332498 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 18:59:32.332498 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 18:59:32.332498 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 18:59:32.332498 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 18:59:32.332498 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 18:59:32.332498 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 18:59:32.332498 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 18:59:32.332498 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 18:59:32.332498 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 18:59:32.332498 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 18:59:32.332498 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 18:59:32.332498 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 18:59:32.334602 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 18:59:32.334602 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 18:59:32.334602 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 18:59:32.334602 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 18:59:32.334602 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 18:59:32.334602 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 18:59:32.334602 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 18:59:32.334602 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 18:59:32.334602 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 18:59:32.334602 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 18:59:32.334602 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 18:59:32.334602 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 18:59:32.334602 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 18:59:32.334602 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 18:59:32.334602 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 18:59:32.334602 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 18:59:32.336122 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 18:59:32.336122 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 18:59:32.336122 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 18:59:32.336122 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 18:59:32.336122 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 18:59:32.336122 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 18:59:32.336122 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 18:59:32.336122 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 18:59:32.336122 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 18:59:32.336122 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 18:59:32.336122 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 02 18:59:32.336122 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 18:59:32.337340 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9142c9-6138-47d2-b1cf-8619a7673016 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:32.337458 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 38/76] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 2 18:59:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 18:59:32.337458 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa10ad52-c5f5-4933-a191-8b819c9910d8 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:32.477863 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa10ad52-c5f5-4933-a191-8b819c9910d8 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:32.484420 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9142c9-6138-47d2-b1cf-8619a7673016 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:32.527031 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4fdfda-b67e-4e78-9091-8d2e1641385c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:32.528033 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4fdfda-b67e-4e78-9091-8d2e1641385c None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:32.528539 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4fdfda-b67e-4e78-9091-8d2e1641385c None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:32.529050 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4fdfda-b67e-4e78-9091-8d2e1641385c None admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:32.530425 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441e2692-6f3a-4be7-8ec2-403abac37c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:32.530591 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4fdfda-b67e-4e78-9091-8d2e1641385c None admin] RBAC: Adding query filter params (name=member) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 18:59:32.531049 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4fdfda-b67e-4e78-9091-8d2e1641385c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:32.531290 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441e2692-6f3a-4be7-8ec2-403abac37c17 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:32.531583 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441e2692-6f3a-4be7-8ec2-403abac37c17 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:32.531866 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441e2692-6f3a-4be7-8ec2-403abac37c17 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:32.533767 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4fdfda-b67e-4e78-9091-8d2e1641385c None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:32.544415 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 39/77] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 2 18:59:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:32.544642 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441e2692-6f3a-4be7-8ec2-403abac37c17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 18:59:32.545263 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4fdfda-b67e-4e78-9091-8d2e1641385c None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:32.553406 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-441e2692-6f3a-4be7-8ec2-403abac37c17 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 02 18:59:32.553406 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 18:59:32.553406 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 18:59:32.553406 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 18:59:32.553406 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 18:59:32.553406 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 18:59:32.553406 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 18:59:32.553406 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 18:59:32.553406 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 18:59:32.553406 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 18:59:32.553406 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 18:59:32.553406 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 18:59:32.553406 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 18:59:32.553406 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 18:59:32.553406 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 18:59:32.553406 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 18:59:32.560185 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 18:59:32.560185 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 18:59:32.560185 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 18:59:32.560185 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 18:59:32.560185 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 18:59:32.560185 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 18:59:32.560185 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 18:59:32.560185 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 18:59:32.560185 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 18:59:32.560185 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 18:59:32.560185 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 18:59:32.560185 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 18:59:32.560185 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 18:59:32.560185 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 18:59:32.560185 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 18:59:32.560185 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 18:59:32.561315 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 18:59:32.561315 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 18:59:32.561315 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 18:59:32.561315 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 18:59:32.561315 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 18:59:32.561315 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 18:59:32.561315 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 18:59:32.561315 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 18:59:32.561315 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 18:59:32.561315 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 18:59:32.561315 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 02 18:59:32.561315 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 18:59:32.561315 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 39/78] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 2 18:59:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 18:59:32.561315 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441e2692-6f3a-4be7-8ec2-403abac37c17 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:32.579236 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:32.607062 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:32.681007 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4fdfda-b67e-4e78-9091-8d2e1641385c None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:32.702401 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441e2692-6f3a-4be7-8ec2-403abac37c17 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:32.716305 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f86185c-25ec-4fc7-b43a-021a6d649dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:32.717122 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f86185c-25ec-4fc7-b43a-021a6d649dfa None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:32.717410 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f86185c-25ec-4fc7-b43a-021a6d649dfa None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:32.717690 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f86185c-25ec-4fc7-b43a-021a6d649dfa None admin] PATH_INFO: `/v3/domains/default` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:32.718988 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f86185c-25ec-4fc7-b43a-021a6d649dfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:32.720542 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f86185c-25ec-4fc7-b43a-021a6d649dfa None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:32.722305 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 40/79] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 18:59:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:32.723164 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f86185c-25ec-4fc7-b43a-021a6d649dfa None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:32.751617 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357ef09f-d572-4a3e-ad01-18cfebf6020c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:32.752277 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357ef09f-d572-4a3e-ad01-18cfebf6020c None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:32.752562 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357ef09f-d572-4a3e-ad01-18cfebf6020c None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:32.752841 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357ef09f-d572-4a3e-ad01-18cfebf6020c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:32.831494 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:32.841042 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357ef09f-d572-4a3e-ad01-18cfebf6020c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:32.842860 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357ef09f-d572-4a3e-ad01-18cfebf6020c None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:32.852011 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f86185c-25ec-4fc7-b43a-021a6d649dfa None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:32.891479 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ba70cf-5de7-41a6-b014-45339cefc51c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:32.892265 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ba70cf-5de7-41a6-b014-45339cefc51c None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:32.892656 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ba70cf-5de7-41a6-b014-45339cefc51c None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:32.893023 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ba70cf-5de7-41a6-b014-45339cefc51c None admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:32.893928 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ba70cf-5de7-41a6-b014-45339cefc51c None admin] RBAC: Adding query filter params (name=member) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 18:59:32.894435 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ba70cf-5de7-41a6-b014-45339cefc51c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:32.896536 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:32.896707 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ba70cf-5de7-41a6-b014-45339cefc51c None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:32.908646 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 41/80] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 2 18:59:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:32.910471 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ba70cf-5de7-41a6-b014-45339cefc51c None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:32.972049 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 40/81] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 2 18:59:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 02 18:59:32.972940 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357ef09f-d572-4a3e-ad01-18cfebf6020c None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:33.064082 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ba70cf-5de7-41a6-b014-45339cefc51c None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:33.089818 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357ef09f-d572-4a3e-ad01-18cfebf6020c None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:33.123923 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08df26b7-13e5-4bdc-9355-1829b06d2bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:33.124807 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08df26b7-13e5-4bdc-9355-1829b06d2bfb None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:33.125100 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08df26b7-13e5-4bdc-9355-1829b06d2bfb None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:33.125738 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08df26b7-13e5-4bdc-9355-1829b06d2bfb None admin] PATH_INFO: `/v3/users/720bc045e0bf473dbe5912892ff7b8e2` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:33.128879 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54719f31-c48b-4352-bd4c-1b29bbaeb441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:33.129546 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54719f31-c48b-4352-bd4c-1b29bbaeb441 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:33.129829 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54719f31-c48b-4352-bd4c-1b29bbaeb441 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:33.130108 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54719f31-c48b-4352-bd4c-1b29bbaeb441 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:33.131526 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54719f31-c48b-4352-bd4c-1b29bbaeb441 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 18:59:33.131836 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54719f31-c48b-4352-bd4c-1b29bbaeb441 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:33.133260 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54719f31-c48b-4352-bd4c-1b29bbaeb441 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:33.142198 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08df26b7-13e5-4bdc-9355-1829b06d2bfb None admin] RBAC: Authorizing `identity:get_user(user_id=720bc045e0bf473dbe5912892ff7b8e2)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:33.142399 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 41/82] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 2 18:59:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:33.143408 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54719f31-c48b-4352-bd4c-1b29bbaeb441 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:33.143723 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08df26b7-13e5-4bdc-9355-1829b06d2bfb None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:33.145477 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 42/83] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 2 18:59:32 2022] GET /identity/v3/users/720bc045e0bf473dbe5912892ff7b8e2 => generated 275 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:33.146814 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08df26b7-13e5-4bdc-9355-1829b06d2bfb None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:33.153245 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:33.162912 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:33.257983 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54719f31-c48b-4352-bd4c-1b29bbaeb441 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:33.259905 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08df26b7-13e5-4bdc-9355-1829b06d2bfb None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:33.294196 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4fc488-fbd7-493e-b264-33dccff7268d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:33.294651 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2fac33-eaae-4fe3-a308-66fffda595db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:33.295159 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4fc488-fbd7-493e-b264-33dccff7268d None admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:33.295412 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4fc488-fbd7-493e-b264-33dccff7268d None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:33.295571 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2fac33-eaae-4fe3-a308-66fffda595db None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:33.295811 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4fc488-fbd7-493e-b264-33dccff7268d None admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:33.295904 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2fac33-eaae-4fe3-a308-66fffda595db None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:33.296124 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2fac33-eaae-4fe3-a308-66fffda595db None admin] PATH_INFO: `/v3/users/720bc045e0bf473dbe5912892ff7b8e2` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:33.296753 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4fc488-fbd7-493e-b264-33dccff7268d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:33.297374 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2fac33-eaae-4fe3-a308-66fffda595db None admin] RBAC: Authorizing `identity:get_user(user_id=720bc045e0bf473dbe5912892ff7b8e2)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:33.298344 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4fc488-fbd7-493e-b264-33dccff7268d None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:33.298845 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2fac33-eaae-4fe3-a308-66fffda595db None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:33.300502 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 43/84] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 2 18:59:33 2022] GET /identity/v3/users/720bc045e0bf473dbe5912892ff7b8e2 => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:33.301275 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2fac33-eaae-4fe3-a308-66fffda595db None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:33.379602 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:33.404225 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2fac33-eaae-4fe3-a308-66fffda595db None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:33.423071 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 42/85] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 2 18:59:33 2022] POST /identity/v3/users => generated 289 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 18:59:33.423953 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4fc488-fbd7-493e-b264-33dccff7268d None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:33.441529 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fd6e28-0690-4ea5-91cb-e87bb169ce9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:33.442446 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd6e28-0690-4ea5-91cb-e87bb169ce9a None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:33.443274 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd6e28-0690-4ea5-91cb-e87bb169ce9a None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:33.443578 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd6e28-0690-4ea5-91cb-e87bb169ce9a None admin] PATH_INFO: `/v3/projects/abffdc3e7a9447068ceac783b7558c81` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:33.445037 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fd6e28-0690-4ea5-91cb-e87bb169ce9a None admin] RBAC: Authorizing `identity:get_project(project_id=abffdc3e7a9447068ceac783b7558c81)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:33.446577 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fd6e28-0690-4ea5-91cb-e87bb169ce9a None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:33.453367 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 44/86] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 2 18:59:33 2022] GET /identity/v3/projects/abffdc3e7a9447068ceac783b7558c81 => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:33.453367 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fd6e28-0690-4ea5-91cb-e87bb169ce9a None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:33.520660 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:33.568410 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4fc488-fbd7-493e-b264-33dccff7268d None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:33.604149 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fd6e28-0690-4ea5-91cb-e87bb169ce9a None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:33.627692 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b346536-9c20-4fda-b587-b20513a052ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:33.629082 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b346536-9c20-4fda-b587-b20513a052ad None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:33.629565 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b346536-9c20-4fda-b587-b20513a052ad None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:33.630095 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b346536-9c20-4fda-b587-b20513a052ad None admin] PATH_INFO: `/v3/users/720bc045e0bf473dbe5912892ff7b8e2` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:33.632550 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b346536-9c20-4fda-b587-b20513a052ad None admin] RBAC: Authorizing `identity:get_user(user_id=720bc045e0bf473dbe5912892ff7b8e2)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:33.635291 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b346536-9c20-4fda-b587-b20513a052ad None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:33.642574 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 43/87] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 2 18:59:33 2022] GET /identity/v3/users/720bc045e0bf473dbe5912892ff7b8e2 => generated 275 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:33.642574 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b346536-9c20-4fda-b587-b20513a052ad None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:33.671281 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a0f9ff-d989-45e3-a5af-b6b0251f2318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:33.672284 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a0f9ff-d989-45e3-a5af-b6b0251f2318 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:33.672714 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a0f9ff-d989-45e3-a5af-b6b0251f2318 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:33.673128 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a0f9ff-d989-45e3-a5af-b6b0251f2318 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:33.691335 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:33.768128 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a0f9ff-d989-45e3-a5af-b6b0251f2318 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:33.770404 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a0f9ff-d989-45e3-a5af-b6b0251f2318 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:33.839806 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b346536-9c20-4fda-b587-b20513a052ad None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:33.875383 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:33.894845 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d220e24-444a-4611-8f5c-36dba802b4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:33.894845 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d220e24-444a-4611-8f5c-36dba802b4df None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:33.894845 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d220e24-444a-4611-8f5c-36dba802b4df None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:33.894845 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d220e24-444a-4611-8f5c-36dba802b4df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:33.925519 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 45/88] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 2 18:59:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 02 18:59:33.928074 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a0f9ff-d989-45e3-a5af-b6b0251f2318 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:33.988290 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:34.040519 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d220e24-444a-4611-8f5c-36dba802b4df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:34.044538 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d220e24-444a-4611-8f5c-36dba802b4df None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:34.065049 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a0f9ff-d989-45e3-a5af-b6b0251f2318 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:34.111906 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fac9b46-b2e2-4fcf-b0fa-290416ef729d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:34.112854 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fac9b46-b2e2-4fcf-b0fa-290416ef729d None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:34.113540 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fac9b46-b2e2-4fcf-b0fa-290416ef729d None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:34.113971 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fac9b46-b2e2-4fcf-b0fa-290416ef729d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:34.115516 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fac9b46-b2e2-4fcf-b0fa-290416ef729d None admin] RBAC: Adding query filter params (role.id=f13fdd0847924014abf1e95594195e78, scope.project.id=abffdc3e7a9447068ceac783b7558c81, user.id=789d0dc0871243f79a6f4f6f602c4f21) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 18:59:34.116071 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fac9b46-b2e2-4fcf-b0fa-290416ef729d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:34.117783 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fac9b46-b2e2-4fcf-b0fa-290416ef729d None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:34.128107 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 46/89] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 2 18:59:33 2022] GET /identity/v3/role_assignments?user.id=789d0dc0871243f79a6f4f6f602c4f21&scope.project.id=abffdc3e7a9447068ceac783b7558c81&role.id=f13fdd0847924014abf1e95594195e78 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:34.129647 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fac9b46-b2e2-4fcf-b0fa-290416ef729d None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:34.216058 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 44/90] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 2 18:59:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 577 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 02 18:59:34.217654 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d220e24-444a-4611-8f5c-36dba802b4df None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:34.272573 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:34.331594 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fac9b46-b2e2-4fcf-b0fa-290416ef729d None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:34.347247 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:34.396614 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383a65f3-0e3a-49ab-9d6e-72af33de63e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:34.399927 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383a65f3-0e3a-49ab-9d6e-72af33de63e4 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:34.399927 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383a65f3-0e3a-49ab-9d6e-72af33de63e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:34.399927 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383a65f3-0e3a-49ab-9d6e-72af33de63e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:34.440423 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d220e24-444a-4611-8f5c-36dba802b4df None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:34.496941 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d616144-feac-4600-84b6-d0be915cfe54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:34.497934 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d616144-feac-4600-84b6-d0be915cfe54 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:34.498523 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d616144-feac-4600-84b6-d0be915cfe54 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:34.498945 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d616144-feac-4600-84b6-d0be915cfe54 None admin] PATH_INFO: `/v3/projects/abffdc3e7a9447068ceac783b7558c81` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:34.500381 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d616144-feac-4600-84b6-d0be915cfe54 None admin] RBAC: Authorizing `identity:get_project(project_id=abffdc3e7a9447068ceac783b7558c81)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:34.502282 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d616144-feac-4600-84b6-d0be915cfe54 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:34.504312 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 45/91] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 2 18:59:34 2022] GET /identity/v3/projects/abffdc3e7a9447068ceac783b7558c81 => generated 298 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:34.505228 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d616144-feac-4600-84b6-d0be915cfe54 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:34.530025 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383a65f3-0e3a-49ab-9d6e-72af33de63e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:34.532018 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383a65f3-0e3a-49ab-9d6e-72af33de63e4 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:34.756982 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:34.773791 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d616144-feac-4600-84b6-d0be915cfe54 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:34.785102 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 47/92] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 2 18:59:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 656 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 02 18:59:34.786597 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383a65f3-0e3a-49ab-9d6e-72af33de63e4 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:34.822758 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:34.916106 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57049365-7ad0-44cd-9775-c491dcdffe8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:34.917371 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57049365-7ad0-44cd-9775-c491dcdffe8a None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:34.917371 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57049365-7ad0-44cd-9775-c491dcdffe8a None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:34.917529 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57049365-7ad0-44cd-9775-c491dcdffe8a None admin] PATH_INFO: `/v3/projects/7e039130ee334ac58867fed690ed99b1` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:34.918740 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57049365-7ad0-44cd-9775-c491dcdffe8a None admin] RBAC: Authorizing `identity:get_project(project_id=7e039130ee334ac58867fed690ed99b1)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:34.920223 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57049365-7ad0-44cd-9775-c491dcdffe8a None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:34.921954 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 46/93] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 2 18:59:34 2022] GET /identity/v3/projects/7e039130ee334ac58867fed690ed99b1 => generated 312 bytes in 417 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:34.922741 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57049365-7ad0-44cd-9775-c491dcdffe8a None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:35.024833 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383a65f3-0e3a-49ab-9d6e-72af33de63e4 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:35.094562 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482655b6-d175-47de-ad41-c0dd4d588774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:35.095742 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482655b6-d175-47de-ad41-c0dd4d588774 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:35.096258 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482655b6-d175-47de-ad41-c0dd4d588774 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:35.096735 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482655b6-d175-47de-ad41-c0dd4d588774 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:35.098121 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482655b6-d175-47de-ad41-c0dd4d588774 None admin] RBAC: Adding query filter params (role.id=77d5301ed78146468d4115b512cc70b7, scope.project.id=abffdc3e7a9447068ceac783b7558c81, user.id=720bc045e0bf473dbe5912892ff7b8e2) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 18:59:35.098628 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482655b6-d175-47de-ad41-c0dd4d588774 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:35.101652 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482655b6-d175-47de-ad41-c0dd4d588774 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:35.113300 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 48/94] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 2 18:59:34 2022] GET /identity/v3/role_assignments?user.id=720bc045e0bf473dbe5912892ff7b8e2&scope.project.id=abffdc3e7a9447068ceac783b7558c81&role.id=77d5301ed78146468d4115b512cc70b7 => generated 259 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:35.115346 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482655b6-d175-47de-ad41-c0dd4d588774 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:35.289258 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:35.297151 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57049365-7ad0-44cd-9775-c491dcdffe8a None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:35.377482 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482655b6-d175-47de-ad41-c0dd4d588774 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:35.420939 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72519792-383d-431f-8718-4dc707d96f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:35.422420 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72519792-383d-431f-8718-4dc707d96f75 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:35.422948 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72519792-383d-431f-8718-4dc707d96f75 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:35.423454 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72519792-383d-431f-8718-4dc707d96f75 None admin] PATH_INFO: `/v3/projects/abffdc3e7a9447068ceac783b7558c81` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:35.425584 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72519792-383d-431f-8718-4dc707d96f75 None admin] RBAC: Authorizing `identity:get_project(project_id=abffdc3e7a9447068ceac783b7558c81)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:35.428429 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72519792-383d-431f-8718-4dc707d96f75 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:35.440201 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 47/95] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 2 18:59:34 2022] GET /identity/v3/projects/abffdc3e7a9447068ceac783b7558c81 => generated 298 bytes in 517 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:35.449504 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72519792-383d-431f-8718-4dc707d96f75 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:35.460784 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcf70ef-e254-46de-80c2-461969cd839d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:35.461482 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf70ef-e254-46de-80c2-461969cd839d None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:35.461775 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf70ef-e254-46de-80c2-461969cd839d None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:35.462061 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf70ef-e254-46de-80c2-461969cd839d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:35.462967 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcf70ef-e254-46de-80c2-461969cd839d None admin] RBAC: Adding query filter params (role.id=77d5301ed78146468d4115b512cc70b7, scope.project.id=7e039130ee334ac58867fed690ed99b1, user.id=720bc045e0bf473dbe5912892ff7b8e2) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 18:59:35.463303 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcf70ef-e254-46de-80c2-461969cd839d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:35.464789 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcf70ef-e254-46de-80c2-461969cd839d None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:35.478711 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 49/96] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 2 18:59:35 2022] GET /identity/v3/role_assignments?user.id=720bc045e0bf473dbe5912892ff7b8e2&scope.project.id=7e039130ee334ac58867fed690ed99b1&role.id=77d5301ed78146468d4115b512cc70b7 => generated 259 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:35.541732 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:35.638795 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72519792-383d-431f-8718-4dc707d96f75 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:35.682697 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64301a26-52b1-4096-aac8-8668ada11970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:35.683416 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64301a26-52b1-4096-aac8-8668ada11970 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:35.683699 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64301a26-52b1-4096-aac8-8668ada11970 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:35.683976 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64301a26-52b1-4096-aac8-8668ada11970 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:35.684839 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64301a26-52b1-4096-aac8-8668ada11970 None admin] RBAC: Adding query filter params (role.id=2e0c054b646e4ed2a0e7e5962f09a50e, scope.project.id=abffdc3e7a9447068ceac783b7558c81, user.id=720bc045e0bf473dbe5912892ff7b8e2) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 18:59:35.685157 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64301a26-52b1-4096-aac8-8668ada11970 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:35.686655 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64301a26-52b1-4096-aac8-8668ada11970 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:35.693770 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 48/97] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 2 18:59:35 2022] GET /identity/v3/role_assignments?user.id=720bc045e0bf473dbe5912892ff7b8e2&scope.project.id=abffdc3e7a9447068ceac783b7558c81&role.id=2e0c054b646e4ed2a0e7e5962f09a50e => generated 259 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:36.564297 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d04a9b4-5e06-447b-983f-40d80854fe47 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:36.564819 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d04a9b4-5e06-447b-983f-40d80854fe47 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:36.564919 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d04a9b4-5e06-447b-983f-40d80854fe47 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:36.566244 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 50/98] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 18:59:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 18:59:36.581076 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d6a95e-5f7e-4d10-9e4e-d39288460d58 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:36.581608 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d6a95e-5f7e-4d10-9e4e-d39288460d58 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:36.581680 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d6a95e-5f7e-4d10-9e4e-d39288460d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:36.604072 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-83d6a95e-5f7e-4d10-9e4e-d39288460d58 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 18:59:36.605327 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-83d6a95e-5f7e-4d10-9e4e-d39288460d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 18:59:36.710319 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 49/99] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 18:59:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 18:59:36.748431 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d04a9b4-5e06-447b-983f-40d80854fe47 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:36.844257 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4466c1e-d328-4826-a4c4-b2cbbb4c8ab0 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:36.844543 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4466c1e-d328-4826-a4c4-b2cbbb4c8ab0 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:36.844752 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4466c1e-d328-4826-a4c4-b2cbbb4c8ab0 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:36.846260 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 50/100] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 18:59:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 18:59:36.856538 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2847f4b-1f56-4fdf-b7a4-da05baa58596 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:36.856819 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2847f4b-1f56-4fdf-b7a4-da05baa58596 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:36.857092 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2847f4b-1f56-4fdf-b7a4-da05baa58596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:36.884582 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-f2847f4b-1f56-4fdf-b7a4-da05baa58596 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 18:59:36.885787 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-f2847f4b-1f56-4fdf-b7a4-da05baa58596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 18:59:36.905605 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d04a9b4-5e06-447b-983f-40d80854fe47 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:36.939773 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:36.965700 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1171597f-253e-4d95-bf83-d117fd3a478c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:36.966519 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1171597f-253e-4d95-bf83-d117fd3a478c None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:36.966832 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1171597f-253e-4d95-bf83-d117fd3a478c None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:36.967113 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1171597f-253e-4d95-bf83-d117fd3a478c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:36.975127 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:36.982290 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1171597f-253e-4d95-bf83-d117fd3a478c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 18:59:36.990961 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-1171597f-253e-4d95-bf83-d117fd3a478c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 02 18:59:36.990961 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 18:59:36.990961 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 18:59:36.990961 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 18:59:36.990961 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 18:59:36.990961 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 18:59:36.990961 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 18:59:36.990961 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 18:59:36.990961 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 18:59:36.990961 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 18:59:36.990961 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 18:59:36.990961 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 18:59:36.990961 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 18:59:36.990961 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 18:59:36.990961 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 18:59:36.990961 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 18:59:37.001664 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 18:59:37.001664 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 18:59:37.001664 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 18:59:37.001664 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 18:59:37.001664 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 18:59:37.001664 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 18:59:37.001664 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 18:59:37.001664 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 18:59:37.001664 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 18:59:37.001664 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 18:59:37.001664 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 18:59:37.001664 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 18:59:37.001664 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 18:59:37.001664 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 18:59:37.001664 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 18:59:37.001664 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 18:59:37.002665 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 18:59:37.002665 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 18:59:37.002665 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 18:59:37.002665 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 18:59:37.002665 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 18:59:37.002665 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 18:59:37.002665 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 18:59:37.002665 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 18:59:37.002665 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 18:59:37.002665 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 18:59:37.002665 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 02 18:59:37.002665 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 18:59:37.002665 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 51/101] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 2 18:59:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 18:59:37.008851 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1171597f-253e-4d95-bf83-d117fd3a478c None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:37.021666 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 51/102] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 18:59:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 18:59:37.046916 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2847f4b-1f56-4fdf-b7a4-da05baa58596 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:37.145009 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1171597f-253e-4d95-bf83-d117fd3a478c None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:37.166706 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2847f4b-1f56-4fdf-b7a4-da05baa58596 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:37.182150 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d7ac18-360c-4ab7-8c9f-309c8cb962cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:37.182876 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7ac18-360c-4ab7-8c9f-309c8cb962cf None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:37.183163 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7ac18-360c-4ab7-8c9f-309c8cb962cf None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:37.183443 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7ac18-360c-4ab7-8c9f-309c8cb962cf None admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:37.184315 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d7ac18-360c-4ab7-8c9f-309c8cb962cf None admin] RBAC: Adding query filter params (name=reader) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 18:59:37.184628 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d7ac18-360c-4ab7-8c9f-309c8cb962cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:37.186154 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d7ac18-360c-4ab7-8c9f-309c8cb962cf None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:37.195783 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 52/103] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 2 18:59:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 18:59:37.201596 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d7ac18-360c-4ab7-8c9f-309c8cb962cf None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:37.207325 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b57fc7-b37d-41ec-ae7c-e0bc3b5b7fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:37.208366 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b57fc7-b37d-41ec-ae7c-e0bc3b5b7fe1 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:37.208827 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b57fc7-b37d-41ec-ae7c-e0bc3b5b7fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:37.209262 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b57fc7-b37d-41ec-ae7c-e0bc3b5b7fe1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:37.210648 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b57fc7-b37d-41ec-ae7c-e0bc3b5b7fe1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:37.212401 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b57fc7-b37d-41ec-ae7c-e0bc3b5b7fe1 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:37.214231 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 52/104] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 18:59:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 18:59:37.220123 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b57fc7-b37d-41ec-ae7c-e0bc3b5b7fe1 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:37.250247 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:37.282044 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:37.332934 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d7ac18-360c-4ab7-8c9f-309c8cb962cf None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:37.335001 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b57fc7-b37d-41ec-ae7c-e0bc3b5b7fe1 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:37.373082 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5348450-9fd4-4899-9c2e-9d786c318d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:37.373885 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5348450-9fd4-4899-9c2e-9d786c318d47 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:37.374173 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5348450-9fd4-4899-9c2e-9d786c318d47 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:37.374449 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5348450-9fd4-4899-9c2e-9d786c318d47 None admin] PATH_INFO: `/v3/users/accce7a27d294457bd72f2a3cc6d8330` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:37.374864 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6ea9cc-f45d-4cb4-82bf-5019967d4efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:37.375886 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ea9cc-f45d-4cb4-82bf-5019967d4efc None admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:37.376638 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ea9cc-f45d-4cb4-82bf-5019967d4efc None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:37.377079 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ea9cc-f45d-4cb4-82bf-5019967d4efc None admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:37.377947 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6ea9cc-f45d-4cb4-82bf-5019967d4efc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:37.379578 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6ea9cc-f45d-4cb4-82bf-5019967d4efc None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:37.389878 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5348450-9fd4-4899-9c2e-9d786c318d47 None admin] RBAC: Authorizing `identity:get_user(user_id=accce7a27d294457bd72f2a3cc6d8330)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:37.391490 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5348450-9fd4-4899-9c2e-9d786c318d47 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:37.393227 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 53/105] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 2 18:59:37 2022] GET /identity/v3/users/accce7a27d294457bd72f2a3cc6d8330 => generated 289 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 18:59:37.402786 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5348450-9fd4-4899-9c2e-9d786c318d47 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:37.483490 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 53/106] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Jun 2 18:59:37 2022] POST /identity/v3/projects => generated 302 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 02 18:59:37.543124 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5348450-9fd4-4899-9c2e-9d786c318d47 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:37.568733 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:37.596334 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03907626-e5f5-47bf-90dc-30e74e8757de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:37.597027 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03907626-e5f5-47bf-90dc-30e74e8757de None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:37.597320 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03907626-e5f5-47bf-90dc-30e74e8757de None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:37.597604 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03907626-e5f5-47bf-90dc-30e74e8757de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:37.698462 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03907626-e5f5-47bf-90dc-30e74e8757de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:37.699966 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03907626-e5f5-47bf-90dc-30e74e8757de None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:37.836611 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 54/107] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 2 18:59:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 02 18:59:37.845033 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6ea9cc-f45d-4cb4-82bf-5019967d4efc None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:37.921454 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a46d9a-6917-4c3c-ad0c-b06eb6d21f9a None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:37.921766 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a46d9a-6917-4c3c-ad0c-b06eb6d21f9a None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:37.922033 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a46d9a-6917-4c3c-ad0c-b06eb6d21f9a None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:37.923617 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 55/108] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 18:59:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 02 18:59:37.930413 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa7eb74-df9e-46b9-8f2e-e0647b6117d5 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:37.930706 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa7eb74-df9e-46b9-8f2e-e0647b6117d5 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:37.931026 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa7eb74-df9e-46b9-8f2e-e0647b6117d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:37.957774 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-3aa7eb74-df9e-46b9-8f2e-e0647b6117d5 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 18:59:37.959239 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-3aa7eb74-df9e-46b9-8f2e-e0647b6117d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 18:59:37.977650 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:38.022173 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:38.072832 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6ea9cc-f45d-4cb4-82bf-5019967d4efc None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:38.102777 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 56/109] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 18:59:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 02 18:59:38.136108 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72933323-14a4-4d9d-ac33-7244f0fa4801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:38.137068 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72933323-14a4-4d9d-ac33-7244f0fa4801 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:38.137383 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72933323-14a4-4d9d-ac33-7244f0fa4801 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:38.137667 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72933323-14a4-4d9d-ac33-7244f0fa4801 None admin] PATH_INFO: `/v3/projects/abffdc3e7a9447068ceac783b7558c81` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:38.138933 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72933323-14a4-4d9d-ac33-7244f0fa4801 None admin] RBAC: Authorizing `identity:get_project(project_id=abffdc3e7a9447068ceac783b7558c81)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:38.140393 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72933323-14a4-4d9d-ac33-7244f0fa4801 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:38.142254 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 54/110] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 2 18:59:37 2022] GET /identity/v3/projects/abffdc3e7a9447068ceac783b7558c81 => generated 298 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:38.144104 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa7eb74-df9e-46b9-8f2e-e0647b6117d5 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:38.154048 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72933323-14a4-4d9d-ac33-7244f0fa4801 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:38.309343 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72933323-14a4-4d9d-ac33-7244f0fa4801 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:38.326961 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa7eb74-df9e-46b9-8f2e-e0647b6117d5 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:38.352071 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643a1a59-85bb-423b-a0fc-ad44ab90a795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:38.353085 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643a1a59-85bb-423b-a0fc-ad44ab90a795 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:38.353620 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643a1a59-85bb-423b-a0fc-ad44ab90a795 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:38.354205 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643a1a59-85bb-423b-a0fc-ad44ab90a795 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:38.355301 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643a1a59-85bb-423b-a0fc-ad44ab90a795 None admin] RBAC: Adding query filter params (role.id=de9eea600632474d8b149397589c3746, scope.project.id=abffdc3e7a9447068ceac783b7558c81, user.id=accce7a27d294457bd72f2a3cc6d8330) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 18:59:38.355820 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643a1a59-85bb-423b-a0fc-ad44ab90a795 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:38.357471 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643a1a59-85bb-423b-a0fc-ad44ab90a795 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:38.363666 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 55/111] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 2 18:59:38 2022] GET /identity/v3/role_assignments?user.id=accce7a27d294457bd72f2a3cc6d8330&scope.project.id=abffdc3e7a9447068ceac783b7558c81&role.id=de9eea600632474d8b149397589c3746 => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:38.394198 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d0ccd0-471d-4c20-88f8-96878d32345b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:38.395065 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d0ccd0-471d-4c20-88f8-96878d32345b None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:38.395353 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d0ccd0-471d-4c20-88f8-96878d32345b None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:38.395627 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d0ccd0-471d-4c20-88f8-96878d32345b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:38.417010 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d0ccd0-471d-4c20-88f8-96878d32345b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 18:59:38.425038 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-52d0ccd0-471d-4c20-88f8-96878d32345b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 02 18:59:38.425038 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 18:59:38.425038 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 18:59:38.425038 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 18:59:38.425038 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 18:59:38.425038 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 18:59:38.425038 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 18:59:38.425038 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 18:59:38.425038 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 18:59:38.425038 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 18:59:38.425038 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 18:59:38.425038 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 18:59:38.425038 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 18:59:38.425038 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 18:59:38.425038 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 18:59:38.425038 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 18:59:38.425929 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 18:59:38.425929 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 18:59:38.425929 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 18:59:38.425929 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 18:59:38.425929 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 18:59:38.425929 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 18:59:38.425929 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 18:59:38.425929 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 18:59:38.425929 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 18:59:38.425929 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 18:59:38.425929 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 18:59:38.425929 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 18:59:38.425929 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 18:59:38.425929 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 18:59:38.425929 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 18:59:38.425929 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 18:59:38.426761 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 18:59:38.426761 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 18:59:38.426761 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 18:59:38.426761 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 18:59:38.426761 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 18:59:38.426761 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 18:59:38.426761 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 18:59:38.426761 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 18:59:38.426761 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 18:59:38.426761 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 18:59:38.426761 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 02 18:59:38.426761 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 18:59:38.426761 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 57/112] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 2 18:59:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 18:59:38.433217 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643a1a59-85bb-423b-a0fc-ad44ab90a795 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:38.459095 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:38.588270 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643a1a59-85bb-423b-a0fc-ad44ab90a795 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:38.659521 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5da205-7400-452d-95e0-6e59a0fc8d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:38.660230 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5da205-7400-452d-95e0-6e59a0fc8d62 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:38.660523 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5da205-7400-452d-95e0-6e59a0fc8d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:38.660810 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5da205-7400-452d-95e0-6e59a0fc8d62 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:38.661608 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5da205-7400-452d-95e0-6e59a0fc8d62 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 18:59:38.661958 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5da205-7400-452d-95e0-6e59a0fc8d62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:38.663414 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5da205-7400-452d-95e0-6e59a0fc8d62 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:38.677207 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 56/113] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 2 18:59:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:38.683914 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d0ccd0-471d-4c20-88f8-96878d32345b None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:38.694886 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:38.907003 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d0ccd0-471d-4c20-88f8-96878d32345b None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:38.950149 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2713a7-bfe3-4af7-b8bc-97dcda6569b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:38.951425 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2713a7-bfe3-4af7-b8bc-97dcda6569b4 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:38.951929 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2713a7-bfe3-4af7-b8bc-97dcda6569b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:38.952394 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2713a7-bfe3-4af7-b8bc-97dcda6569b4 None admin] PATH_INFO: `/v3/users/789d0dc0871243f79a6f4f6f602c4f21` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:38.953844 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2713a7-bfe3-4af7-b8bc-97dcda6569b4 None admin] RBAC: Authorizing `identity:get_user(user_id=789d0dc0871243f79a6f4f6f602c4f21)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:38.955540 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2713a7-bfe3-4af7-b8bc-97dcda6569b4 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:38.957827 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 58/114] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 2 18:59:38 2022] GET /identity/v3/users/789d0dc0871243f79a6f4f6f602c4f21 => generated 247 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:38.963667 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5da205-7400-452d-95e0-6e59a0fc8d62 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:39.083061 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:39.142619 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5da205-7400-452d-95e0-6e59a0fc8d62 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:39.197843 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f9cb43-ba49-441d-86da-bc6ecc3aad91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:39.198665 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f9cb43-ba49-441d-86da-bc6ecc3aad91 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:39.198985 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f9cb43-ba49-441d-86da-bc6ecc3aad91 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:39.199267 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f9cb43-ba49-441d-86da-bc6ecc3aad91 None admin] PATH_INFO: `/v3/projects/abffdc3e7a9447068ceac783b7558c81` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:39.200603 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f9cb43-ba49-441d-86da-bc6ecc3aad91 None admin] RBAC: Authorizing `identity:get_project(project_id=abffdc3e7a9447068ceac783b7558c81)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:39.202151 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f9cb43-ba49-441d-86da-bc6ecc3aad91 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:39.204222 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 57/115] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 2 18:59:38 2022] GET /identity/v3/projects/abffdc3e7a9447068ceac783b7558c81 => generated 298 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:39.211144 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2713a7-bfe3-4af7-b8bc-97dcda6569b4 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:39.308175 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:39.343595 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2713a7-bfe3-4af7-b8bc-97dcda6569b4 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:39.385836 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08ddc93-2056-476b-9714-3cd95964295a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:39.386786 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ddc93-2056-476b-9714-3cd95964295a None admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:39.387086 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ddc93-2056-476b-9714-3cd95964295a None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:39.387371 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ddc93-2056-476b-9714-3cd95964295a None admin] PATH_INFO: `/v3/projects/abffdc3e7a9447068ceac783b7558c81/users/789d0dc0871243f79a6f4f6f602c4f21/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:39.389384 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08ddc93-2056-476b-9714-3cd95964295a None admin] RBAC: Authorizing `identity:create_grant(project_id=abffdc3e7a9447068ceac783b7558c81, user_id=789d0dc0871243f79a6f4f6f602c4f21, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:39.390882 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08ddc93-2056-476b-9714-3cd95964295a None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:39.482804 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 59/116] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 2 18:59:39 2022] PUT /identity/v3/projects/abffdc3e7a9447068ceac783b7558c81/users/789d0dc0871243f79a6f4f6f602c4f21/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 18:59:39.567126 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b7d3a2-b8c5-450a-bbe0-1a5b91e46c48 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:39.567461 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b7d3a2-b8c5-450a-bbe0-1a5b91e46c48 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:39.567708 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b7d3a2-b8c5-450a-bbe0-1a5b91e46c48 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:39.569919 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 58/117] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 18:59:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 18:59:39.588946 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55886cdb-e3ee-465f-81a1-fd373745d535 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:39.589259 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55886cdb-e3ee-465f-81a1-fd373745d535 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:39.589534 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55886cdb-e3ee-465f-81a1-fd373745d535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:39.612274 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-55886cdb-e3ee-465f-81a1-fd373745d535 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 18:59:39.613504 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-55886cdb-e3ee-465f-81a1-fd373745d535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 18:59:39.682972 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615888fd-35d6-48ca-bf86-153331df3c34 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:39.683270 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615888fd-35d6-48ca-bf86-153331df3c34 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:39.683552 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615888fd-35d6-48ca-bf86-153331df3c34 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:39.685089 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 59/118] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 18:59:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 02 18:59:39.690577 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03703e18-c067-4501-abb6-83105c983bc6 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:39.690956 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03703e18-c067-4501-abb6-83105c983bc6 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:39.691252 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03703e18-c067-4501-abb6-83105c983bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:39.697603 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 60/119] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 18:59:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 18:59:39.721670 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55886cdb-e3ee-465f-81a1-fd373745d535 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:39.734048 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-03703e18-c067-4501-abb6-83105c983bc6 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 18:59:39.735289 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-03703e18-c067-4501-abb6-83105c983bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 18:59:39.801856 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:39.860893 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55886cdb-e3ee-465f-81a1-fd373745d535 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:39.879636 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 60/120] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 18:59:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 18:59:39.889705 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1ce35f-d3f5-4762-bc12-55d44ce593ca None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:39.890002 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1ce35f-d3f5-4762-bc12-55d44ce593ca None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:39.890273 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1ce35f-d3f5-4762-bc12-55d44ce593ca None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:39.891897 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 61/121] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 18:59:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 02 18:59:39.897763 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aac698-f598-4425-9867-88e2ab37f64f None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:39.898029 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aac698-f598-4425-9867-88e2ab37f64f None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:39.898312 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aac698-f598-4425-9867-88e2ab37f64f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:39.901391 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efa0696-cb4b-4985-b4d7-c41aa54a444f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:39.902220 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efa0696-cb4b-4985-b4d7-c41aa54a444f None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:39.902622 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efa0696-cb4b-4985-b4d7-c41aa54a444f None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:39.902942 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efa0696-cb4b-4985-b4d7-c41aa54a444f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:39.915934 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efa0696-cb4b-4985-b4d7-c41aa54a444f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 18:59:39.917799 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-04aac698-f598-4425-9867-88e2ab37f64f None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 18:59:39.919065 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-04aac698-f598-4425-9867-88e2ab37f64f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 18:59:39.924448 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-0efa0696-cb4b-4985-b4d7-c41aa54a444f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 02 18:59:39.924448 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 18:59:39.924448 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 18:59:39.924448 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 18:59:39.924448 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 18:59:39.924448 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 18:59:39.924448 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 18:59:39.924448 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 18:59:39.924448 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 18:59:39.924448 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 18:59:39.924448 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 18:59:39.924448 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 18:59:39.924448 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 18:59:39.924448 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 18:59:39.924448 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 18:59:39.924448 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 18:59:39.925304 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 18:59:39.925304 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 18:59:39.925304 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 18:59:39.925304 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 18:59:39.925304 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 18:59:39.925304 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 18:59:39.925304 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 18:59:39.925304 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 18:59:39.925304 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 18:59:39.925304 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 18:59:39.925304 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 18:59:39.925304 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 18:59:39.925304 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 18:59:39.925304 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 18:59:39.925304 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 18:59:39.925304 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 18:59:39.926025 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 18:59:39.926025 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 18:59:39.926025 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 18:59:39.926025 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 18:59:39.926025 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 18:59:39.926025 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 18:59:39.926025 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 18:59:39.926025 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 18:59:39.926025 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 18:59:39.926025 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 18:59:39.926025 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 02 18:59:39.926025 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 18:59:39.926025 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 61/122] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 2 18:59:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 18:59:39.927199 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efa0696-cb4b-4985-b4d7-c41aa54a444f None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:39.984952 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:39.991187 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 62/123] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 18:59:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 02 18:59:39.995651 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aac698-f598-4425-9867-88e2ab37f64f None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:40.049408 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efa0696-cb4b-4985-b4d7-c41aa54a444f None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:40.076014 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:40.089501 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2eef54-30ed-4028-9162-34712961e9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:40.090325 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2eef54-30ed-4028-9162-34712961e9f0 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:40.090625 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2eef54-30ed-4028-9162-34712961e9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:40.090940 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2eef54-30ed-4028-9162-34712961e9f0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:40.104289 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2eef54-30ed-4028-9162-34712961e9f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 18:59:40.121415 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-bb2eef54-30ed-4028-9162-34712961e9f0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 02 18:59:40.121415 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 18:59:40.121415 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 18:59:40.121415 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 18:59:40.121415 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 18:59:40.121415 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 18:59:40.121415 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 18:59:40.121415 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 18:59:40.121415 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 18:59:40.121415 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 18:59:40.121415 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 18:59:40.121415 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 18:59:40.121415 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 18:59:40.121415 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 18:59:40.121415 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 18:59:40.121415 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 18:59:40.129124 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 18:59:40.129124 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 18:59:40.129124 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 18:59:40.129124 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 18:59:40.129124 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 18:59:40.129124 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 18:59:40.129124 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 18:59:40.129124 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 18:59:40.129124 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 18:59:40.129124 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 18:59:40.129124 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 18:59:40.129124 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 18:59:40.129124 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 18:59:40.129124 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 18:59:40.129124 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 18:59:40.129124 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 18:59:40.131119 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 18:59:40.131119 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 18:59:40.131119 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 18:59:40.131119 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 18:59:40.131119 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 18:59:40.131119 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 18:59:40.131119 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 18:59:40.131119 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 18:59:40.131119 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 18:59:40.131119 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 18:59:40.131119 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 02 18:59:40.131119 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 18:59:40.131119 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 62/124] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 2 18:59:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 18:59:40.131119 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2eef54-30ed-4028-9162-34712961e9f0 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:40.140328 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aac698-f598-4425-9867-88e2ab37f64f None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:40.158006 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b7e532-0fe5-4872-9b6e-054637080bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:40.158708 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b7e532-0fe5-4872-9b6e-054637080bbe None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:40.159041 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b7e532-0fe5-4872-9b6e-054637080bbe None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:40.159325 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b7e532-0fe5-4872-9b6e-054637080bbe None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:40.160103 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b7e532-0fe5-4872-9b6e-054637080bbe None admin] RBAC: Adding query filter params (name=member) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 18:59:40.160419 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b7e532-0fe5-4872-9b6e-054637080bbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:40.162384 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b7e532-0fe5-4872-9b6e-054637080bbe None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:40.171504 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 63/125] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 2 18:59:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:40.172355 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b7e532-0fe5-4872-9b6e-054637080bbe None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:40.233992 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:40.239945 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2eef54-30ed-4028-9162-34712961e9f0 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:40.277179 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b7e532-0fe5-4872-9b6e-054637080bbe None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:40.278478 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c436067-d303-4642-8135-c1c8244ddb36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:40.279342 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c436067-d303-4642-8135-c1c8244ddb36 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:40.279633 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c436067-d303-4642-8135-c1c8244ddb36 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:40.279917 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c436067-d303-4642-8135-c1c8244ddb36 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:40.293915 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c436067-d303-4642-8135-c1c8244ddb36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 18:59:40.310950 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-4c436067-d303-4642-8135-c1c8244ddb36 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 02 18:59:40.310950 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 18:59:40.310950 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 18:59:40.310950 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 18:59:40.310950 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 18:59:40.310950 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 18:59:40.310950 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 18:59:40.310950 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 18:59:40.310950 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 18:59:40.310950 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 18:59:40.310950 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 18:59:40.310950 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 18:59:40.310950 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 18:59:40.310950 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 18:59:40.310950 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 18:59:40.310950 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 18:59:40.313422 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 18:59:40.313422 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 18:59:40.313422 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 18:59:40.313422 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 18:59:40.313422 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 18:59:40.313422 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 18:59:40.313422 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 18:59:40.313422 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 18:59:40.313422 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 18:59:40.313422 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 18:59:40.313422 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 18:59:40.313422 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 18:59:40.313422 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 18:59:40.313422 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 18:59:40.313422 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 18:59:40.313422 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 18:59:40.315322 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 18:59:40.315322 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 18:59:40.315322 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 18:59:40.315322 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 18:59:40.315322 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 18:59:40.315322 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 18:59:40.315322 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 18:59:40.315322 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 18:59:40.315322 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 18:59:40.315322 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 18:59:40.315322 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 02 18:59:40.315322 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 18:59:40.315322 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 63/126] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 2 18:59:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 18:59:40.315322 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c436067-d303-4642-8135-c1c8244ddb36 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:40.316957 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9d031b-3c2e-4941-a73d-7b8cbca12029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:40.316957 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d031b-3c2e-4941-a73d-7b8cbca12029 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:40.316957 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d031b-3c2e-4941-a73d-7b8cbca12029 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:40.316957 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d031b-3c2e-4941-a73d-7b8cbca12029 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:40.317473 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9d031b-3c2e-4941-a73d-7b8cbca12029 None admin] RBAC: Adding query filter params (name=member) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 18:59:40.317473 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9d031b-3c2e-4941-a73d-7b8cbca12029 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:40.322786 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9d031b-3c2e-4941-a73d-7b8cbca12029 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:40.328416 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 64/127] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 2 18:59:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:40.329265 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9d031b-3c2e-4941-a73d-7b8cbca12029 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:40.433306 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:40.439821 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9d031b-3c2e-4941-a73d-7b8cbca12029 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:40.466129 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:40.509187 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c436067-d303-4642-8135-c1c8244ddb36 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:40.511461 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5542fc-a046-44f5-9da7-2c71f0887d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:40.512174 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5542fc-a046-44f5-9da7-2c71f0887d43 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:40.512464 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5542fc-a046-44f5-9da7-2c71f0887d43 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:40.512748 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5542fc-a046-44f5-9da7-2c71f0887d43 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:40.513523 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5542fc-a046-44f5-9da7-2c71f0887d43 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 18:59:40.513836 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5542fc-a046-44f5-9da7-2c71f0887d43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:40.515290 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5542fc-a046-44f5-9da7-2c71f0887d43 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:40.534924 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 65/128] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 2 18:59:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:40.535859 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5542fc-a046-44f5-9da7-2c71f0887d43 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:40.547083 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe00b2d0-2f5f-4930-ae23-2a404e70cc94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:40.548168 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe00b2d0-2f5f-4930-ae23-2a404e70cc94 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:40.548470 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe00b2d0-2f5f-4930-ae23-2a404e70cc94 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:40.548758 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe00b2d0-2f5f-4930-ae23-2a404e70cc94 None admin] PATH_INFO: `/v3/users/720bc045e0bf473dbe5912892ff7b8e2` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:40.550002 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe00b2d0-2f5f-4930-ae23-2a404e70cc94 None admin] RBAC: Authorizing `identity:get_user(user_id=720bc045e0bf473dbe5912892ff7b8e2)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:40.551533 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe00b2d0-2f5f-4930-ae23-2a404e70cc94 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:40.553752 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 64/129] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 2 18:59:40 2022] GET /identity/v3/users/720bc045e0bf473dbe5912892ff7b8e2 => generated 275 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:40.554576 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe00b2d0-2f5f-4930-ae23-2a404e70cc94 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:40.681186 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5542fc-a046-44f5-9da7-2c71f0887d43 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:40.707582 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe00b2d0-2f5f-4930-ae23-2a404e70cc94 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:40.720523 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:40.722445 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26428a6-54a3-4786-8a29-e0eedb037419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:40.723568 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26428a6-54a3-4786-8a29-e0eedb037419 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:40.728156 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26428a6-54a3-4786-8a29-e0eedb037419 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:40.728156 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26428a6-54a3-4786-8a29-e0eedb037419 None admin] PATH_INFO: `/v3/users/720bc045e0bf473dbe5912892ff7b8e2` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:40.728156 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26428a6-54a3-4786-8a29-e0eedb037419 None admin] RBAC: Authorizing `identity:get_user(user_id=720bc045e0bf473dbe5912892ff7b8e2)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:40.728156 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26428a6-54a3-4786-8a29-e0eedb037419 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:40.730531 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 66/130] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 2 18:59:40 2022] GET /identity/v3/users/720bc045e0bf473dbe5912892ff7b8e2 => generated 275 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:40.731706 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26428a6-54a3-4786-8a29-e0eedb037419 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:40.769912 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8972b80c-3009-40ce-a755-2e833e7fccc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:40.772542 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8972b80c-3009-40ce-a755-2e833e7fccc1 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:40.773408 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8972b80c-3009-40ce-a755-2e833e7fccc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:40.774086 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8972b80c-3009-40ce-a755-2e833e7fccc1 None admin] PATH_INFO: `/v3/users/720bc045e0bf473dbe5912892ff7b8e2` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:40.777021 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8972b80c-3009-40ce-a755-2e833e7fccc1 None admin] RBAC: Authorizing `identity:get_user(user_id=720bc045e0bf473dbe5912892ff7b8e2)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:40.780257 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8972b80c-3009-40ce-a755-2e833e7fccc1 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:40.784620 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 65/131] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 2 18:59:40 2022] GET /identity/v3/users/720bc045e0bf473dbe5912892ff7b8e2 => generated 275 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:40.786437 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8972b80c-3009-40ce-a755-2e833e7fccc1 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:40.866397 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:40.907815 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8972b80c-3009-40ce-a755-2e833e7fccc1 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:40.950947 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf1e23e-307f-4655-b437-e4f258611870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:40.952752 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf1e23e-307f-4655-b437-e4f258611870 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:40.953226 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf1e23e-307f-4655-b437-e4f258611870 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:40.953705 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf1e23e-307f-4655-b437-e4f258611870 None admin] PATH_INFO: `/v3/projects/7e039130ee334ac58867fed690ed99b1` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:40.955256 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf1e23e-307f-4655-b437-e4f258611870 None admin] RBAC: Authorizing `identity:get_project(project_id=7e039130ee334ac58867fed690ed99b1)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:40.957074 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf1e23e-307f-4655-b437-e4f258611870 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:40.960717 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 66/132] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 2 18:59:40 2022] GET /identity/v3/projects/7e039130ee334ac58867fed690ed99b1 => generated 312 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:40.962413 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf1e23e-307f-4655-b437-e4f258611870 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:40.992518 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26428a6-54a3-4786-8a29-e0eedb037419 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:41.028994 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:41.059438 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de947219-06b2-4e50-a56b-5ef5ed766cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:41.060647 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de947219-06b2-4e50-a56b-5ef5ed766cd3 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:41.061382 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de947219-06b2-4e50-a56b-5ef5ed766cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:41.062553 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de947219-06b2-4e50-a56b-5ef5ed766cd3 None admin] PATH_INFO: `/v3/projects/abffdc3e7a9447068ceac783b7558c81` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:41.064057 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de947219-06b2-4e50-a56b-5ef5ed766cd3 None admin] RBAC: Authorizing `identity:get_project(project_id=abffdc3e7a9447068ceac783b7558c81)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:41.068898 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de947219-06b2-4e50-a56b-5ef5ed766cd3 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:41.070897 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 67/133] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 2 18:59:40 2022] GET /identity/v3/projects/abffdc3e7a9447068ceac783b7558c81 => generated 298 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:41.074821 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de947219-06b2-4e50-a56b-5ef5ed766cd3 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:41.138100 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf1e23e-307f-4655-b437-e4f258611870 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:41.184700 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c16e72-eeaf-4183-ba36-7825b19a5eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:41.185577 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c16e72-eeaf-4183-ba36-7825b19a5eef None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:41.185864 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c16e72-eeaf-4183-ba36-7825b19a5eef None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:41.186302 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c16e72-eeaf-4183-ba36-7825b19a5eef None admin] PATH_INFO: `/v3/projects/abffdc3e7a9447068ceac783b7558c81` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:41.187865 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c16e72-eeaf-4183-ba36-7825b19a5eef None admin] RBAC: Authorizing `identity:get_project(project_id=abffdc3e7a9447068ceac783b7558c81)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:41.189339 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c16e72-eeaf-4183-ba36-7825b19a5eef None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:41.191603 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 67/134] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 2 18:59:40 2022] GET /identity/v3/projects/abffdc3e7a9447068ceac783b7558c81 => generated 298 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:41.195856 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c16e72-eeaf-4183-ba36-7825b19a5eef None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:41.210741 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de947219-06b2-4e50-a56b-5ef5ed766cd3 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:41.233080 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:41.249587 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbfa945-77c0-4aa7-bf86-66b384dcc092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:41.250952 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbfa945-77c0-4aa7-bf86-66b384dcc092 None admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:41.251696 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbfa945-77c0-4aa7-bf86-66b384dcc092 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:41.252170 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbfa945-77c0-4aa7-bf86-66b384dcc092 None admin] PATH_INFO: `/v3/projects/7e039130ee334ac58867fed690ed99b1/users/720bc045e0bf473dbe5912892ff7b8e2/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:41.254453 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbfa945-77c0-4aa7-bf86-66b384dcc092 None admin] RBAC: Authorizing `identity:create_grant(project_id=7e039130ee334ac58867fed690ed99b1, user_id=720bc045e0bf473dbe5912892ff7b8e2, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:41.256327 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbfa945-77c0-4aa7-bf86-66b384dcc092 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:41.314800 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c16e72-eeaf-4183-ba36-7825b19a5eef None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:41.330376 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:41.350067 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364da431-149c-47a7-bde4-c89559ce2086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:41.351033 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364da431-149c-47a7-bde4-c89559ce2086 None admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:41.351788 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364da431-149c-47a7-bde4-c89559ce2086 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:41.351788 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364da431-149c-47a7-bde4-c89559ce2086 None admin] PATH_INFO: `/v3/projects/abffdc3e7a9447068ceac783b7558c81/users/720bc045e0bf473dbe5912892ff7b8e2/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:41.353548 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364da431-149c-47a7-bde4-c89559ce2086 None admin] RBAC: Authorizing `identity:create_grant(project_id=abffdc3e7a9447068ceac783b7558c81, user_id=720bc045e0bf473dbe5912892ff7b8e2, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:41.355014 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364da431-149c-47a7-bde4-c89559ce2086 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:41.387827 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 68/135] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 2 18:59:41 2022] PUT /identity/v3/projects/7e039130ee334ac58867fed690ed99b1/users/720bc045e0bf473dbe5912892ff7b8e2/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 18:59:41.387827 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f82758-b28f-4ede-9323-d9ffa26d6b6f None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:41.387827 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f82758-b28f-4ede-9323-d9ffa26d6b6f None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:41.387827 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f82758-b28f-4ede-9323-d9ffa26d6b6f None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:41.391941 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 69/136] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 18:59:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 18:59:41.392867 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f82758-b28f-4ede-9323-d9ffa26d6b6f None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:41.459521 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 68/137] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 2 18:59:41 2022] PUT /identity/v3/projects/abffdc3e7a9447068ceac783b7558c81/users/720bc045e0bf473dbe5912892ff7b8e2/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 18:59:41.460929 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633b4153-c7d0-44dd-aa43-c982abae78ad None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:41.461339 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633b4153-c7d0-44dd-aa43-c982abae78ad None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:41.461613 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633b4153-c7d0-44dd-aa43-c982abae78ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:41.509207 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-633b4153-c7d0-44dd-aa43-c982abae78ad None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 18:59:41.512298 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-633b4153-c7d0-44dd-aa43-c982abae78ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 18:59:41.590403 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f82758-b28f-4ede-9323-d9ffa26d6b6f None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:41.642551 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 69/138] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 18:59:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 18:59:41.668182 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633b4153-c7d0-44dd-aa43-c982abae78ad None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:41.695938 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef24ebe6-447b-40ec-8642-ebe4bc82ff88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:41.697330 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef24ebe6-447b-40ec-8642-ebe4bc82ff88 None admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:41.697829 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef24ebe6-447b-40ec-8642-ebe4bc82ff88 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:41.698308 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef24ebe6-447b-40ec-8642-ebe4bc82ff88 None admin] PATH_INFO: `/v3/projects/abffdc3e7a9447068ceac783b7558c81/users/720bc045e0bf473dbe5912892ff7b8e2/roles/2e0c054b646e4ed2a0e7e5962f09a50e` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:41.702827 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef24ebe6-447b-40ec-8642-ebe4bc82ff88 None admin] RBAC: Authorizing `identity:create_grant(project_id=abffdc3e7a9447068ceac783b7558c81, user_id=720bc045e0bf473dbe5912892ff7b8e2, role_id=2e0c054b646e4ed2a0e7e5962f09a50e)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:41.708646 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef24ebe6-447b-40ec-8642-ebe4bc82ff88 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:41.811211 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 70/139] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 2 18:59:41 2022] PUT /identity/v3/projects/abffdc3e7a9447068ceac783b7558c81/users/720bc045e0bf473dbe5912892ff7b8e2/roles/2e0c054b646e4ed2a0e7e5962f09a50e => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 18:59:41.897466 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633b4153-c7d0-44dd-aa43-c982abae78ad None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:41.904733 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:41.984137 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4a2ce1-e1dc-4bef-be58-426780e2935d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:41.985661 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4a2ce1-e1dc-4bef-be58-426780e2935d None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:41.986179 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4a2ce1-e1dc-4bef-be58-426780e2935d None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:41.986694 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4a2ce1-e1dc-4bef-be58-426780e2935d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:41.989160 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4a2ce1-e1dc-4bef-be58-426780e2935d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:41.992679 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4a2ce1-e1dc-4bef-be58-426780e2935d None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:41.996032 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 70/140] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 18:59:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:42.007938 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef24ebe6-447b-40ec-8642-ebe4bc82ff88 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:42.041667 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86b3a7c-5654-4f22-8f13-5de89f063dc8 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:42.042184 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86b3a7c-5654-4f22-8f13-5de89f063dc8 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:42.042707 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86b3a7c-5654-4f22-8f13-5de89f063dc8 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:42.045525 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 71/141] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 18:59:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 18:59:42.056710 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182602cc-2f74-44bc-906d-20c12ea4c0e0 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:42.057206 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182602cc-2f74-44bc-906d-20c12ea4c0e0 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:42.057676 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182602cc-2f74-44bc-906d-20c12ea4c0e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:42.072709 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:42.091073 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-182602cc-2f74-44bc-906d-20c12ea4c0e0 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 18:59:42.094242 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-182602cc-2f74-44bc-906d-20c12ea4c0e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 18:59:42.204335 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef24ebe6-447b-40ec-8642-ebe4bc82ff88 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:42.253226 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 72/142] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 18:59:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 18:59:42.269135 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96255ed2-c791-42e8-b551-475d93738ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:42.269871 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96255ed2-c791-42e8-b551-475d93738ede None admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:42.270168 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96255ed2-c791-42e8-b551-475d93738ede None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:42.270454 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96255ed2-c791-42e8-b551-475d93738ede None admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:42.271244 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96255ed2-c791-42e8-b551-475d93738ede None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:42.272923 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96255ed2-c791-42e8-b551-475d93738ede None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:42.298470 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182602cc-2f74-44bc-906d-20c12ea4c0e0 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:42.377408 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 71/143] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 2 18:59:42 2022] POST /identity/v3/users => generated 283 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 18:59:42.454133 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182602cc-2f74-44bc-906d-20c12ea4c0e0 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:42.456305 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:42.499829 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b7a17d-7e1b-49b4-91db-a530ceefa6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:42.500652 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7a17d-7e1b-49b4-91db-a530ceefa6c4 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:42.500990 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7a17d-7e1b-49b4-91db-a530ceefa6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:42.501541 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7a17d-7e1b-49b4-91db-a530ceefa6c4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:42.517703 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b7a17d-7e1b-49b4-91db-a530ceefa6c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 18:59:42.530734 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-c1b7a17d-7e1b-49b4-91db-a530ceefa6c4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 02 18:59:42.530734 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 18:59:42.530734 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 18:59:42.530734 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 18:59:42.530734 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 18:59:42.530734 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 18:59:42.530734 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 18:59:42.530734 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 18:59:42.530734 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 18:59:42.530734 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 18:59:42.530734 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 18:59:42.530734 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 18:59:42.530734 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 18:59:42.530734 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 18:59:42.530734 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 18:59:42.530734 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 18:59:42.531669 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 18:59:42.531669 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 18:59:42.531669 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 18:59:42.531669 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 18:59:42.531669 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 18:59:42.531669 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 18:59:42.531669 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 18:59:42.531669 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 18:59:42.531669 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 18:59:42.531669 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 18:59:42.531669 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 18:59:42.531669 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 18:59:42.531669 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 18:59:42.531669 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 18:59:42.531669 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 18:59:42.531669 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 18:59:42.532409 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 18:59:42.532409 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 18:59:42.532409 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 18:59:42.532409 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 18:59:42.532409 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 18:59:42.532409 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 18:59:42.532409 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 18:59:42.532409 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 18:59:42.532409 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 18:59:42.532409 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 18:59:42.532409 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 02 18:59:42.532409 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 18:59:42.532409 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 73/144] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 2 18:59:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 02 18:59:42.543270 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96255ed2-c791-42e8-b551-475d93738ede None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:42.648015 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da10c514-f9d5-4405-92de-1222883e75cc None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:42.648330 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da10c514-f9d5-4405-92de-1222883e75cc None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:42.648589 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da10c514-f9d5-4405-92de-1222883e75cc None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:42.650002 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 74/145] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 18:59:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 02 18:59:42.655921 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e922360-c504-41dc-97c5-ce79605ca12d None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:42.656206 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e922360-c504-41dc-97c5-ce79605ca12d None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:42.656490 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e922360-c504-41dc-97c5-ce79605ca12d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:42.691955 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-1e922360-c504-41dc-97c5-ce79605ca12d None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 18:59:42.693745 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-1e922360-c504-41dc-97c5-ce79605ca12d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 18:59:42.709812 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:42.734727 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96255ed2-c791-42e8-b551-475d93738ede None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:42.837156 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6c7fa2-971f-449d-955e-c022374fa893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:42.837853 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c7fa2-971f-449d-955e-c022374fa893 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:42.838146 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c7fa2-971f-449d-955e-c022374fa893 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:42.838689 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c7fa2-971f-449d-955e-c022374fa893 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:42.840359 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6c7fa2-971f-449d-955e-c022374fa893 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 18:59:42.840680 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6c7fa2-971f-449d-955e-c022374fa893 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:42.842822 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6c7fa2-971f-449d-955e-c022374fa893 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:42.850560 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 75/146] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 18:59:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 02 18:59:42.862142 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 72/147] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 2 18:59:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 18:59:42.871097 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e922360-c504-41dc-97c5-ce79605ca12d None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:42.874569 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6c7fa2-971f-449d-955e-c022374fa893 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:43.049434 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:43.065973 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e922360-c504-41dc-97c5-ce79605ca12d None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:43.113357 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6c7fa2-971f-449d-955e-c022374fa893 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:43.142724 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7f7b26-1e73-4c81-848d-f49db4769c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:43.143995 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f7b26-1e73-4c81-848d-f49db4769c11 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:43.144468 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f7b26-1e73-4c81-848d-f49db4769c11 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:43.144968 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f7b26-1e73-4c81-848d-f49db4769c11 None admin] PATH_INFO: `/v3/users/accce7a27d294457bd72f2a3cc6d8330` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:43.146517 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7f7b26-1e73-4c81-848d-f49db4769c11 None admin] RBAC: Authorizing `identity:get_user(user_id=accce7a27d294457bd72f2a3cc6d8330)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:43.148812 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7f7b26-1e73-4c81-848d-f49db4769c11 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:43.151022 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 76/148] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 2 18:59:42 2022] GET /identity/v3/users/accce7a27d294457bd72f2a3cc6d8330 => generated 289 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:43.158972 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7f7b26-1e73-4c81-848d-f49db4769c11 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:43.209461 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913ee988-916d-462d-abc7-26edf225039c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:43.210286 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913ee988-916d-462d-abc7-26edf225039c None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:43.210577 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913ee988-916d-462d-abc7-26edf225039c None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:43.210896 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913ee988-916d-462d-abc7-26edf225039c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:43.236924 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913ee988-916d-462d-abc7-26edf225039c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 18:59:43.252137 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-913ee988-916d-462d-abc7-26edf225039c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 02 18:59:43.252137 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 18:59:43.252137 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 18:59:43.252137 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 18:59:43.252137 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 18:59:43.252137 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 18:59:43.252137 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 18:59:43.252137 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 18:59:43.252137 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 18:59:43.252137 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 18:59:43.252137 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 18:59:43.252137 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 18:59:43.252137 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 18:59:43.252137 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 18:59:43.252137 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 18:59:43.252137 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 18:59:43.253025 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 18:59:43.253025 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 18:59:43.253025 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 18:59:43.253025 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 18:59:43.253025 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 18:59:43.253025 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 18:59:43.253025 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 18:59:43.253025 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 18:59:43.253025 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 18:59:43.253025 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 18:59:43.253025 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 18:59:43.253025 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 18:59:43.253025 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 18:59:43.253025 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 18:59:43.253025 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 18:59:43.253025 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 18:59:43.253784 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 18:59:43.253784 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 18:59:43.253784 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 18:59:43.253784 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 18:59:43.253784 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 18:59:43.253784 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 18:59:43.253784 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 18:59:43.253784 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 18:59:43.253784 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 18:59:43.253784 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 18:59:43.253784 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 02 18:59:43.253784 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 18:59:43.254357 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 73/149] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 2 18:59:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 380 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 18:59:43.259951 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913ee988-916d-462d-abc7-26edf225039c None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:43.277383 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7f7b26-1e73-4c81-848d-f49db4769c11 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:43.293181 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:43.315670 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8bb204-66a9-4287-b715-057ef4fbb9dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:43.316471 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8bb204-66a9-4287-b715-057ef4fbb9dc None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:43.316752 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8bb204-66a9-4287-b715-057ef4fbb9dc None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:43.317029 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8bb204-66a9-4287-b715-057ef4fbb9dc None admin] PATH_INFO: `/v3/projects/abffdc3e7a9447068ceac783b7558c81` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:43.318437 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8bb204-66a9-4287-b715-057ef4fbb9dc None admin] RBAC: Authorizing `identity:get_project(project_id=abffdc3e7a9447068ceac783b7558c81)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:43.320051 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8bb204-66a9-4287-b715-057ef4fbb9dc None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:43.321960 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 77/150] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 2 18:59:43 2022] GET /identity/v3/projects/abffdc3e7a9447068ceac783b7558c81 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:43.327566 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8bb204-66a9-4287-b715-057ef4fbb9dc None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:43.357932 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:43.448442 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8bb204-66a9-4287-b715-057ef4fbb9dc None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:43.481377 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913ee988-916d-462d-abc7-26edf225039c None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:43.512044 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9686d7a-3939-4b4d-9e79-0fec6c70bbed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:43.512964 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9686d7a-3939-4b4d-9e79-0fec6c70bbed None admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:43.513249 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9686d7a-3939-4b4d-9e79-0fec6c70bbed None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:43.513531 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9686d7a-3939-4b4d-9e79-0fec6c70bbed None admin] PATH_INFO: `/v3/projects/abffdc3e7a9447068ceac783b7558c81/users/accce7a27d294457bd72f2a3cc6d8330/roles/de9eea600632474d8b149397589c3746` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:43.520449 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9686d7a-3939-4b4d-9e79-0fec6c70bbed None admin] RBAC: Authorizing `identity:create_grant(project_id=abffdc3e7a9447068ceac783b7558c81, user_id=accce7a27d294457bd72f2a3cc6d8330, role_id=de9eea600632474d8b149397589c3746)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:43.521992 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9686d7a-3939-4b4d-9e79-0fec6c70bbed None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:43.564404 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6a9a24-dcd8-492e-8bdd-8307c45afbe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:43.565098 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a9a24-dcd8-492e-8bdd-8307c45afbe2 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:43.565385 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a9a24-dcd8-492e-8bdd-8307c45afbe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:43.565662 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a9a24-dcd8-492e-8bdd-8307c45afbe2 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:43.566487 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6a9a24-dcd8-492e-8bdd-8307c45afbe2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 18:59:43.566764 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6a9a24-dcd8-492e-8bdd-8307c45afbe2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:43.568851 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6a9a24-dcd8-492e-8bdd-8307c45afbe2 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:43.586186 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 74/151] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 2 18:59:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 18:59:43.591699 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6a9a24-dcd8-492e-8bdd-8307c45afbe2 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:43.635414 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 78/152] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 2 18:59:43 2022] PUT /identity/v3/projects/abffdc3e7a9447068ceac783b7558c81/users/accce7a27d294457bd72f2a3cc6d8330/roles/de9eea600632474d8b149397589c3746 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 18:59:43.746389 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:43.772475 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6a9a24-dcd8-492e-8bdd-8307c45afbe2 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:43.818265 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ba172c-a258-46d3-9524-1c3205c89280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:43.819050 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba172c-a258-46d3-9524-1c3205c89280 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:43.819342 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba172c-a258-46d3-9524-1c3205c89280 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:43.819625 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba172c-a258-46d3-9524-1c3205c89280 None admin] PATH_INFO: `/v3/users/789d0dc0871243f79a6f4f6f602c4f21` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:43.821016 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ba172c-a258-46d3-9524-1c3205c89280 None admin] RBAC: Authorizing `identity:get_user(user_id=789d0dc0871243f79a6f4f6f602c4f21)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:43.822613 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ba172c-a258-46d3-9524-1c3205c89280 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:43.824556 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 75/153] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 2 18:59:43 2022] GET /identity/v3/users/789d0dc0871243f79a6f4f6f602c4f21 => generated 247 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 18:59:43.830642 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9686d7a-3939-4b4d-9e79-0fec6c70bbed None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:43.957257 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:43.971819 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9686d7a-3939-4b4d-9e79-0fec6c70bbed None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:44.012412 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e929186-adc3-40ee-acf9-d80f7069ed98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:44.013136 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e929186-adc3-40ee-acf9-d80f7069ed98 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:44.013428 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e929186-adc3-40ee-acf9-d80f7069ed98 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:44.013737 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e929186-adc3-40ee-acf9-d80f7069ed98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:44.098711 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e929186-adc3-40ee-acf9-d80f7069ed98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:44.100220 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e929186-adc3-40ee-acf9-d80f7069ed98 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:44.226851 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:44.227400 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 79/154] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 2 18:59:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 02 18:59:44.231283 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ba172c-a258-46d3-9524-1c3205c89280 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:44.370717 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ba172c-a258-46d3-9524-1c3205c89280 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:44.410904 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fa8140-2543-43e8-b73c-219c09226710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:44.411823 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fa8140-2543-43e8-b73c-219c09226710 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:44.412111 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fa8140-2543-43e8-b73c-219c09226710 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:44.412391 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fa8140-2543-43e8-b73c-219c09226710 None admin] PATH_INFO: `/v3/projects/abffdc3e7a9447068ceac783b7558c81` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:44.413682 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fa8140-2543-43e8-b73c-219c09226710 None admin] RBAC: Authorizing `identity:get_project(project_id=abffdc3e7a9447068ceac783b7558c81)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:44.415189 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fa8140-2543-43e8-b73c-219c09226710 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:44.417294 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 76/155] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 2 18:59:44 2022] GET /identity/v3/projects/abffdc3e7a9447068ceac783b7558c81 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:44.422628 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e929186-adc3-40ee-acf9-d80f7069ed98 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:44.570185 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e929186-adc3-40ee-acf9-d80f7069ed98 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:44.611762 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97afa99-5631-495f-8224-e37cf97b6aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:44.612828 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97afa99-5631-495f-8224-e37cf97b6aee None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:44.613313 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97afa99-5631-495f-8224-e37cf97b6aee None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:44.613778 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97afa99-5631-495f-8224-e37cf97b6aee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:44.614834 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97afa99-5631-495f-8224-e37cf97b6aee None admin] RBAC: Adding query filter params (role.id=f13fdd0847924014abf1e95594195e78, scope.project.id=abffdc3e7a9447068ceac783b7558c81, user.id=789d0dc0871243f79a6f4f6f602c4f21) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 18:59:44.615449 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97afa99-5631-495f-8224-e37cf97b6aee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:44.617102 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97afa99-5631-495f-8224-e37cf97b6aee None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:44.623964 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 80/156] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 2 18:59:44 2022] GET /identity/v3/role_assignments?user.id=789d0dc0871243f79a6f4f6f602c4f21&scope.project.id=abffdc3e7a9447068ceac783b7558c81&role.id=f13fdd0847924014abf1e95594195e78 => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:45.878968 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42148140-4cab-472b-91f8-f1f4b99333c7 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:45.879547 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42148140-4cab-472b-91f8-f1f4b99333c7 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:45.879547 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42148140-4cab-472b-91f8-f1f4b99333c7 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:45.882129 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 77/157] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 18:59:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 18:59:45.888673 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7124db-cf39-4fb0-97fc-8bd0acb9fa1b None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:45.889209 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7124db-cf39-4fb0-97fc-8bd0acb9fa1b None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:45.889209 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7124db-cf39-4fb0-97fc-8bd0acb9fa1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:45.911625 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-7d7124db-cf39-4fb0-97fc-8bd0acb9fa1b None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 18:59:45.913643 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-7d7124db-cf39-4fb0-97fc-8bd0acb9fa1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 18:59:45.968316 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff1460d-a857-4637-b55f-e098cadbafea None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:45.968639 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff1460d-a857-4637-b55f-e098cadbafea None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:45.968924 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff1460d-a857-4637-b55f-e098cadbafea None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:45.970464 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 78/158] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 18:59:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 02 18:59:45.976525 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41f59ac-a41e-48f9-b178-04f1a5472efe None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:45.976819 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41f59ac-a41e-48f9-b178-04f1a5472efe None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:45.977101 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41f59ac-a41e-48f9-b178-04f1a5472efe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:45.982551 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:46.008288 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-f41f59ac-a41e-48f9-b178-04f1a5472efe None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 18:59:46.009486 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-f41f59ac-a41e-48f9-b178-04f1a5472efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 18:59:46.020528 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 81/159] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 18:59:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 02 18:59:46.040365 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:46.111856 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7124db-cf39-4fb0-97fc-8bd0acb9fa1b None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:46.152414 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 79/160] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 18:59:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 02 18:59:46.206906 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41f59ac-a41e-48f9-b178-04f1a5472efe None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:46.381035 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7124db-cf39-4fb0-97fc-8bd0acb9fa1b None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:46.471399 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1e73e2-17ff-4f82-bd48-9a72a705e37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:46.472225 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1e73e2-17ff-4f82-bd48-9a72a705e37d None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:46.472512 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1e73e2-17ff-4f82-bd48-9a72a705e37d None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:46.472786 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1e73e2-17ff-4f82-bd48-9a72a705e37d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:46.493709 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1e73e2-17ff-4f82-bd48-9a72a705e37d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 18:59:46.500713 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41f59ac-a41e-48f9-b178-04f1a5472efe None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:46.503177 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-5a1e73e2-17ff-4f82-bd48-9a72a705e37d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 02 18:59:46.503177 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 18:59:46.503177 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 18:59:46.503177 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 18:59:46.503177 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 18:59:46.503177 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 18:59:46.503177 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 18:59:46.503177 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 18:59:46.503177 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 18:59:46.503177 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 18:59:46.503177 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 18:59:46.503177 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 18:59:46.503177 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 18:59:46.503177 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 18:59:46.503177 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 18:59:46.503177 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 18:59:46.504089 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 18:59:46.504089 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 18:59:46.504089 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 18:59:46.504089 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 18:59:46.504089 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 18:59:46.504089 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 18:59:46.504089 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 18:59:46.504089 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 18:59:46.504089 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 18:59:46.504089 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 18:59:46.504089 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 18:59:46.504089 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 18:59:46.504089 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 18:59:46.504089 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 18:59:46.504089 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 18:59:46.504089 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 18:59:46.504826 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 18:59:46.504826 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 18:59:46.504826 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 18:59:46.504826 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 18:59:46.504826 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 18:59:46.504826 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 18:59:46.504826 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 18:59:46.504826 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 18:59:46.504826 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 18:59:46.504826 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 18:59:46.504826 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 02 18:59:46.504826 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 18:59:46.509073 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 82/161] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 2 18:59:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 398 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 02 18:59:46.510725 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac82392-682f-49d2-8de7-510dd6ef786d None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:46.511247 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac82392-682f-49d2-8de7-510dd6ef786d None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:46.511714 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac82392-682f-49d2-8de7-510dd6ef786d None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:46.513243 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 83/162] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 18:59:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 18:59:46.519498 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c214732-528d-44c6-b2aa-04bc1e123758 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:46.520074 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c214732-528d-44c6-b2aa-04bc1e123758 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:46.520912 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c214732-528d-44c6-b2aa-04bc1e123758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:46.549696 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-1c214732-528d-44c6-b2aa-04bc1e123758 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 18:59:46.551449 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-1c214732-528d-44c6-b2aa-04bc1e123758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 18:59:46.615237 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:46.667059 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 84/163] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 18:59:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 02 18:59:46.667960 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c214732-528d-44c6-b2aa-04bc1e123758 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:46.668570 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae00a3e0-b9fa-4dad-b681-b9c638f6c862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:46.669526 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae00a3e0-b9fa-4dad-b681-b9c638f6c862 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:46.669817 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae00a3e0-b9fa-4dad-b681-b9c638f6c862 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:46.670097 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae00a3e0-b9fa-4dad-b681-b9c638f6c862 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:46.687746 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae00a3e0-b9fa-4dad-b681-b9c638f6c862 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 18:59:46.704067 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-ae00a3e0-b9fa-4dad-b681-b9c638f6c862 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 02 18:59:46.704067 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 18:59:46.704067 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 18:59:46.704067 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 18:59:46.704067 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 18:59:46.704067 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 18:59:46.704067 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 18:59:46.704067 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 18:59:46.704067 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 18:59:46.704067 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 18:59:46.704067 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 18:59:46.704067 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 18:59:46.704067 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 18:59:46.704067 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 18:59:46.704067 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 18:59:46.704067 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 18:59:46.705023 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 18:59:46.705023 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 18:59:46.705023 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 18:59:46.705023 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 18:59:46.705023 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 18:59:46.705023 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 18:59:46.705023 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 18:59:46.705023 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 18:59:46.705023 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 18:59:46.705023 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 18:59:46.705023 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 18:59:46.705023 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 18:59:46.705023 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 18:59:46.705023 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 18:59:46.705023 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 18:59:46.705023 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 18:59:46.705804 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 18:59:46.705804 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 18:59:46.705804 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 18:59:46.705804 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 18:59:46.705804 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 18:59:46.705804 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 18:59:46.705804 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 18:59:46.705804 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 18:59:46.705804 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 18:59:46.705804 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 18:59:46.705804 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 02 18:59:46.705804 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 18:59:46.705804 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 80/164] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 2 18:59:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 499 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 02 18:59:46.719861 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae00a3e0-b9fa-4dad-b681-b9c638f6c862 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:46.733234 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:46.898360 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c214732-528d-44c6-b2aa-04bc1e123758 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:47.029248 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae00a3e0-b9fa-4dad-b681-b9c638f6c862 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:47.036654 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d7278f-66c3-4e67-995e-3f0c289eb060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:47.038225 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d7278f-66c3-4e67-995e-3f0c289eb060 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:47.038775 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d7278f-66c3-4e67-995e-3f0c289eb060 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:47.039412 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d7278f-66c3-4e67-995e-3f0c289eb060 None admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:47.040412 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d7278f-66c3-4e67-995e-3f0c289eb060 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 18:59:47.040923 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d7278f-66c3-4e67-995e-3f0c289eb060 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:47.042546 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d7278f-66c3-4e67-995e-3f0c289eb060 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:47.062380 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 85/165] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 2 18:59:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 395 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:47.063877 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d7278f-66c3-4e67-995e-3f0c289eb060 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:47.093057 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5ac713-b307-4fa4-90a1-280f50729ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:47.093744 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ac713-b307-4fa4-90a1-280f50729ca0 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:47.094030 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ac713-b307-4fa4-90a1-280f50729ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:47.094303 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ac713-b307-4fa4-90a1-280f50729ca0 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:47.095126 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5ac713-b307-4fa4-90a1-280f50729ca0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 18:59:47.095434 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5ac713-b307-4fa4-90a1-280f50729ca0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:47.096854 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5ac713-b307-4fa4-90a1-280f50729ca0 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:47.116187 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 81/166] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 2 18:59:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 397 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:47.117088 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5ac713-b307-4fa4-90a1-280f50729ca0 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:47.144262 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:47.230633 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:47.246597 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d7278f-66c3-4e67-995e-3f0c289eb060 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:47.268449 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5ac713-b307-4fa4-90a1-280f50729ca0 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:47.297986 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7929a533-c87f-474e-b7c0-6aca343c3a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:47.298732 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7929a533-c87f-474e-b7c0-6aca343c3a4c None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:47.299100 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7929a533-c87f-474e-b7c0-6aca343c3a4c None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:47.299385 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7929a533-c87f-474e-b7c0-6aca343c3a4c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:47.310218 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cd3337-af21-44da-bd76-0c4e3ac5abe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:47.311505 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd3337-af21-44da-bd76-0c4e3ac5abe7 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:47.311963 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd3337-af21-44da-bd76-0c4e3ac5abe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:47.312430 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd3337-af21-44da-bd76-0c4e3ac5abe7 None admin] PATH_INFO: `/v3/users/720bc045e0bf473dbe5912892ff7b8e2` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:47.313325 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7929a533-c87f-474e-b7c0-6aca343c3a4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 18:59:47.314121 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cd3337-af21-44da-bd76-0c4e3ac5abe7 None admin] RBAC: Authorizing `identity:get_user(user_id=720bc045e0bf473dbe5912892ff7b8e2)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:47.315786 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cd3337-af21-44da-bd76-0c4e3ac5abe7 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:47.317597 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 82/167] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 2 18:59:47 2022] GET /identity/v3/users/720bc045e0bf473dbe5912892ff7b8e2 => generated 275 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:47.321283 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-7929a533-c87f-474e-b7c0-6aca343c3a4c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 02 18:59:47.321283 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 18:59:47.321283 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 18:59:47.321283 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 18:59:47.321283 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 18:59:47.321283 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 18:59:47.321283 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 18:59:47.321283 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 18:59:47.321283 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 18:59:47.321283 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 18:59:47.321283 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 18:59:47.321283 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 18:59:47.321283 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 18:59:47.321283 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 18:59:47.321283 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 18:59:47.321283 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 18:59:47.322132 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 18:59:47.322132 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 18:59:47.322132 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 18:59:47.322132 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 18:59:47.322132 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 18:59:47.322132 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 18:59:47.322132 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 18:59:47.322132 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 18:59:47.322132 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 18:59:47.322132 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 18:59:47.322132 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 18:59:47.322132 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 18:59:47.322132 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 18:59:47.322132 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 18:59:47.322132 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 18:59:47.322132 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 18:59:47.323164 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 18:59:47.323164 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 18:59:47.323164 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 18:59:47.323164 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 18:59:47.323164 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 18:59:47.323164 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 18:59:47.323164 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 18:59:47.323164 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 18:59:47.323164 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 18:59:47.323164 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 18:59:47.323164 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 02 18:59:47.323164 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 18:59:47.324846 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a596a2-32a4-4db0-b6bc-8a71acb02213 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:47.325200 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a596a2-32a4-4db0-b6bc-8a71acb02213 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:47.325579 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a596a2-32a4-4db0-b6bc-8a71acb02213 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:47.325924 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 86/168] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 2 18:59:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 18:59:47.326977 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7929a533-c87f-474e-b7c0-6aca343c3a4c None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:47.327652 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 83/169] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 18:59:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 18:59:47.328868 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed593433-d6ff-4879-8959-d1d218f55406 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:47.329455 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed593433-d6ff-4879-8959-d1d218f55406 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:47.329994 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed593433-d6ff-4879-8959-d1d218f55406 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:47.331904 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 84/170] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 18:59:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 18:59:47.332823 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed593433-d6ff-4879-8959-d1d218f55406 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:47.463432 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed593433-d6ff-4879-8959-d1d218f55406 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:47.473067 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:47.483272 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7929a533-c87f-474e-b7c0-6aca343c3a4c None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:47.507718 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16d0632-3cfb-49f9-a423-39e7f3a2a1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:47.508985 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d0632-3cfb-49f9-a423-39e7f3a2a1b7 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:47.509283 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d0632-3cfb-49f9-a423-39e7f3a2a1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:47.509563 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d0632-3cfb-49f9-a423-39e7f3a2a1b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:47.513390 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:47.523655 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71c6e62-1650-429c-b04d-c40f55af11ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:47.524731 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71c6e62-1650-429c-b04d-c40f55af11ff None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:47.525011 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71c6e62-1650-429c-b04d-c40f55af11ff None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:47.525286 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71c6e62-1650-429c-b04d-c40f55af11ff None admin] PATH_INFO: `/v3/users/720bc045e0bf473dbe5912892ff7b8e2` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:47.526494 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71c6e62-1650-429c-b04d-c40f55af11ff None admin] RBAC: Authorizing `identity:get_user(user_id=720bc045e0bf473dbe5912892ff7b8e2)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:47.528007 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71c6e62-1650-429c-b04d-c40f55af11ff None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:47.529816 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 87/171] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 2 18:59:47 2022] GET /identity/v3/users/720bc045e0bf473dbe5912892ff7b8e2 => generated 275 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:47.530472 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71c6e62-1650-429c-b04d-c40f55af11ff None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:47.582425 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16d0632-3cfb-49f9-a423-39e7f3a2a1b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:47.583977 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16d0632-3cfb-49f9-a423-39e7f3a2a1b7 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:47.638681 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71c6e62-1650-429c-b04d-c40f55af11ff None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:47.679151 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7d6b0c-361e-4c15-93d5-0286927aeed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:47.679841 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d6b0c-361e-4c15-93d5-0286927aeed0 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:47.680118 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d6b0c-361e-4c15-93d5-0286927aeed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:47.680396 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d6b0c-361e-4c15-93d5-0286927aeed0 None admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:47.681161 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7d6b0c-361e-4c15-93d5-0286927aeed0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 18:59:47.681472 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7d6b0c-361e-4c15-93d5-0286927aeed0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:47.682886 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7d6b0c-361e-4c15-93d5-0286927aeed0 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:47.696146 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 85/172] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 2 18:59:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 02 18:59:47.696345 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 88/173] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 2 18:59:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:47.696464 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658b35a-4f03-41fc-9afe-85cb60720446 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:47.696464 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658b35a-4f03-41fc-9afe-85cb60720446 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:47.696611 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb95057-0b85-4efb-94a5-fd319b563249 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:47.696687 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658b35a-4f03-41fc-9afe-85cb60720446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:47.696762 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb95057-0b85-4efb-94a5-fd319b563249 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:47.696762 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb95057-0b85-4efb-94a5-fd319b563249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:47.712914 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-f658b35a-4f03-41fc-9afe-85cb60720446 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 18:59:47.714100 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-f658b35a-4f03-41fc-9afe-85cb60720446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 18:59:47.717825 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-1bb95057-0b85-4efb-94a5-fd319b563249 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 18:59:47.719065 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-1bb95057-0b85-4efb-94a5-fd319b563249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 18:59:47.737984 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:47.747959 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:47.789102 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 86/174] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 18:59:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 18:59:47.790972 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8082dc1d-2cde-42f8-b5bc-41132ca52d94 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:47.791463 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8082dc1d-2cde-42f8-b5bc-41132ca52d94 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:47.791931 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8082dc1d-2cde-42f8-b5bc-41132ca52d94 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:47.793688 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 87/175] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 18:59:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 18:59:47.794621 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8082dc1d-2cde-42f8-b5bc-41132ca52d94 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:47.795014 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 89/176] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 18:59:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 18:59:47.795932 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb95057-0b85-4efb-94a5-fd319b563249 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:47.905854 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb95057-0b85-4efb-94a5-fd319b563249 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:47.914778 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8082dc1d-2cde-42f8-b5bc-41132ca52d94 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:47.941616 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad379fa-4fc2-4018-b2e0-478e2ec40b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:47.942494 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad379fa-4fc2-4018-b2e0-478e2ec40b7e None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:47.942797 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad379fa-4fc2-4018-b2e0-478e2ec40b7e None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:47.943077 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad379fa-4fc2-4018-b2e0-478e2ec40b7e None admin] PATH_INFO: `/v3/projects/abffdc3e7a9447068ceac783b7558c81` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:47.944807 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad379fa-4fc2-4018-b2e0-478e2ec40b7e None admin] RBAC: Authorizing `identity:get_project(project_id=abffdc3e7a9447068ceac783b7558c81)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:47.946302 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad379fa-4fc2-4018-b2e0-478e2ec40b7e None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:47.948083 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 90/177] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 2 18:59:47 2022] GET /identity/v3/projects/abffdc3e7a9447068ceac783b7558c81 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:47.951667 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad379fa-4fc2-4018-b2e0-478e2ec40b7e None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:47.954276 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aba3d3-3ba8-4057-9e02-40ceeb5c8dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:47.954980 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aba3d3-3ba8-4057-9e02-40ceeb5c8dac None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:47.955268 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aba3d3-3ba8-4057-9e02-40ceeb5c8dac None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:47.955551 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aba3d3-3ba8-4057-9e02-40ceeb5c8dac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:47.978487 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:47.991759 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:48.027467 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aba3d3-3ba8-4057-9e02-40ceeb5c8dac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:48.029009 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aba3d3-3ba8-4057-9e02-40ceeb5c8dac None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:48.059964 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad379fa-4fc2-4018-b2e0-478e2ec40b7e None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:48.095077 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda0bfa4-eed2-4ba9-8f78-6d16e729cb62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:48.095957 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda0bfa4-eed2-4ba9-8f78-6d16e729cb62 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:48.096239 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda0bfa4-eed2-4ba9-8f78-6d16e729cb62 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:48.096513 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda0bfa4-eed2-4ba9-8f78-6d16e729cb62 None admin] PATH_INFO: `/v3/users/720bc045e0bf473dbe5912892ff7b8e2` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:48.097739 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda0bfa4-eed2-4ba9-8f78-6d16e729cb62 None admin] RBAC: Authorizing `identity:get_user(user_id=720bc045e0bf473dbe5912892ff7b8e2)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:48.099250 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda0bfa4-eed2-4ba9-8f78-6d16e729cb62 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:48.101692 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 91/178] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 2 18:59:47 2022] GET /identity/v3/users/720bc045e0bf473dbe5912892ff7b8e2 => generated 275 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:48.102914 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a6be92-ffef-4e6f-9f2f-9a21b5a9b3da None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:48.103220 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a6be92-ffef-4e6f-9f2f-9a21b5a9b3da None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:48.103490 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a6be92-ffef-4e6f-9f2f-9a21b5a9b3da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:48.121010 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-33a6be92-ffef-4e6f-9f2f-9a21b5a9b3da None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 18:59:48.122233 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-33a6be92-ffef-4e6f-9f2f-9a21b5a9b3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 18:59:48.141172 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 88/179] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 2 18:59:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 02 18:59:48.142542 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e7159d-4eab-4e70-a7c9-37ec3f8798af None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:48.142856 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e7159d-4eab-4e70-a7c9-37ec3f8798af None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:48.143136 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e7159d-4eab-4e70-a7c9-37ec3f8798af None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:48.144475 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 89/180] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 18:59:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 18:59:48.145215 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e7159d-4eab-4e70-a7c9-37ec3f8798af None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:48.195290 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 92/181] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 18:59:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 18:59:48.196055 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a6be92-ffef-4e6f-9f2f-9a21b5a9b3da None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:48.217713 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:48.227573 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:48.262320 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e7159d-4eab-4e70-a7c9-37ec3f8798af None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:48.297088 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a6be92-ffef-4e6f-9f2f-9a21b5a9b3da None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:48.298508 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa81e024-b021-4b30-8a04-92906d4f29e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:48.299433 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa81e024-b021-4b30-8a04-92906d4f29e2 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:48.299794 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa81e024-b021-4b30-8a04-92906d4f29e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:48.300150 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa81e024-b021-4b30-8a04-92906d4f29e2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:48.312718 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa81e024-b021-4b30-8a04-92906d4f29e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 18:59:48.321692 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-aa81e024-b021-4b30-8a04-92906d4f29e2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 02 18:59:48.321692 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 18:59:48.321692 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 18:59:48.321692 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 18:59:48.321692 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 18:59:48.321692 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 18:59:48.321692 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 18:59:48.321692 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 18:59:48.321692 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 18:59:48.321692 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 18:59:48.321692 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 18:59:48.321692 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 18:59:48.321692 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 18:59:48.321692 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 18:59:48.321692 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 18:59:48.321692 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 18:59:48.322646 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 18:59:48.322646 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 18:59:48.322646 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 18:59:48.322646 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 18:59:48.322646 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 18:59:48.322646 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 18:59:48.322646 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 18:59:48.322646 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 18:59:48.322646 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 18:59:48.322646 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 18:59:48.322646 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 18:59:48.322646 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 18:59:48.322646 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 18:59:48.322646 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 18:59:48.322646 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 18:59:48.322646 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 18:59:48.328991 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 18:59:48.328991 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 18:59:48.328991 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 18:59:48.328991 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 18:59:48.328991 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 18:59:48.328991 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 18:59:48.328991 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 18:59:48.328991 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 18:59:48.328991 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 18:59:48.328991 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 18:59:48.328991 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 02 18:59:48.328991 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 18:59:48.328991 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 90/182] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 2 18:59:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 18:59:48.328991 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa81e024-b021-4b30-8a04-92906d4f29e2 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:48.336442 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac493797-20fb-4e28-88be-69339105db47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:48.337323 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac493797-20fb-4e28-88be-69339105db47 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:48.337610 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac493797-20fb-4e28-88be-69339105db47 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:48.337893 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac493797-20fb-4e28-88be-69339105db47 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:48.350126 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac493797-20fb-4e28-88be-69339105db47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 18:59:48.358165 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-ac493797-20fb-4e28-88be-69339105db47 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 02 18:59:48.358165 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 18:59:48.358165 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 18:59:48.358165 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 18:59:48.358165 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 18:59:48.358165 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 18:59:48.358165 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 18:59:48.358165 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 18:59:48.358165 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 18:59:48.358165 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 18:59:48.358165 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 18:59:48.358165 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 18:59:48.358165 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 18:59:48.358165 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 18:59:48.358165 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 18:59:48.358165 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 18:59:48.359142 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 18:59:48.359142 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 18:59:48.359142 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 18:59:48.359142 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 18:59:48.359142 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 18:59:48.359142 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 18:59:48.359142 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 18:59:48.359142 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 18:59:48.359142 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 18:59:48.359142 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 18:59:48.359142 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 18:59:48.359142 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 18:59:48.359142 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 18:59:48.359142 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 18:59:48.359142 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 18:59:48.359142 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 18:59:48.359935 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 18:59:48.359935 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 18:59:48.359935 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 18:59:48.359935 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 18:59:48.359935 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 18:59:48.359935 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 18:59:48.359935 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 18:59:48.359935 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 18:59:48.359935 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 18:59:48.359935 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 18:59:48.359935 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 02 18:59:48.359935 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 18:59:48.359935 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 93/183] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 2 18:59:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 18:59:48.360578 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac493797-20fb-4e28-88be-69339105db47 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:48.432243 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa81e024-b021-4b30-8a04-92906d4f29e2 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:48.443298 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:48.457481 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:48.462852 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac493797-20fb-4e28-88be-69339105db47 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:48.466030 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8db562-44c8-45eb-b1b0-7da9445d1b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:48.466699 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8db562-44c8-45eb-b1b0-7da9445d1b78 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:48.467017 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8db562-44c8-45eb-b1b0-7da9445d1b78 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:48.467299 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8db562-44c8-45eb-b1b0-7da9445d1b78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:48.468226 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8db562-44c8-45eb-b1b0-7da9445d1b78 None admin] RBAC: Adding query filter params (role.id=2e0c054b646e4ed2a0e7e5962f09a50e, scope.project.id=abffdc3e7a9447068ceac783b7558c81, user.id=720bc045e0bf473dbe5912892ff7b8e2) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 18:59:48.468548 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8db562-44c8-45eb-b1b0-7da9445d1b78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:48.470139 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8db562-44c8-45eb-b1b0-7da9445d1b78 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:48.477022 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 91/184] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 2 18:59:48 2022] GET /identity/v3/role_assignments?user.id=720bc045e0bf473dbe5912892ff7b8e2&scope.project.id=abffdc3e7a9447068ceac783b7558c81&role.id=2e0c054b646e4ed2a0e7e5962f09a50e => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:48.477938 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8db562-44c8-45eb-b1b0-7da9445d1b78 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:48.513842 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a20174-1abe-4700-bd5f-ec54b3ee53ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:48.514528 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a20174-1abe-4700-bd5f-ec54b3ee53ac None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:48.514852 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a20174-1abe-4700-bd5f-ec54b3ee53ac None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:48.515137 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a20174-1abe-4700-bd5f-ec54b3ee53ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:48.598227 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a20174-1abe-4700-bd5f-ec54b3ee53ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:48.598476 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8db562-44c8-45eb-b1b0-7da9445d1b78 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:48.600309 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a20174-1abe-4700-bd5f-ec54b3ee53ac None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:48.637103 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848bda3e-77b9-4e3b-8969-4ca8544a1925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:48.637995 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848bda3e-77b9-4e3b-8969-4ca8544a1925 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:48.638279 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848bda3e-77b9-4e3b-8969-4ca8544a1925 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:48.638559 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848bda3e-77b9-4e3b-8969-4ca8544a1925 None admin] PATH_INFO: `/v3/projects/7e039130ee334ac58867fed690ed99b1` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:48.639921 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848bda3e-77b9-4e3b-8969-4ca8544a1925 None admin] RBAC: Authorizing `identity:get_project(project_id=7e039130ee334ac58867fed690ed99b1)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:48.641415 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848bda3e-77b9-4e3b-8969-4ca8544a1925 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:48.643359 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 92/185] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 2 18:59:48 2022] GET /identity/v3/projects/7e039130ee334ac58867fed690ed99b1 => generated 312 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:48.644651 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afc87ba-9558-49a6-a5c5-0615c59fae2e None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:48.644939 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afc87ba-9558-49a6-a5c5-0615c59fae2e None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:48.645219 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afc87ba-9558-49a6-a5c5-0615c59fae2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:48.674909 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-2afc87ba-9558-49a6-a5c5-0615c59fae2e None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 18:59:48.676167 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-2afc87ba-9558-49a6-a5c5-0615c59fae2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 18:59:48.708065 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:48.737801 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 94/186] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 2 18:59:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 02 18:59:48.739040 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a20174-1abe-4700-bd5f-ec54b3ee53ac None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:48.752476 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:48.774841 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 93/187] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 18:59:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 18:59:48.774841 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afc87ba-9558-49a6-a5c5-0615c59fae2e None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:48.866424 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a20174-1abe-4700-bd5f-ec54b3ee53ac None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:48.884286 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afc87ba-9558-49a6-a5c5-0615c59fae2e None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:48.904666 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eed8b3c-ea25-4d3c-91ae-abfc9d180eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:48.905475 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eed8b3c-ea25-4d3c-91ae-abfc9d180eaf None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:48.905756 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eed8b3c-ea25-4d3c-91ae-abfc9d180eaf None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:48.906027 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eed8b3c-ea25-4d3c-91ae-abfc9d180eaf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:48.917026 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eed8b3c-ea25-4d3c-91ae-abfc9d180eaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 18:59:48.918970 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f94664a-5b43-4726-b366-33fe8ab3fc46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:48.919633 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f94664a-5b43-4726-b366-33fe8ab3fc46 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:48.919912 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f94664a-5b43-4726-b366-33fe8ab3fc46 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:48.920184 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f94664a-5b43-4726-b366-33fe8ab3fc46 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:48.920933 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f94664a-5b43-4726-b366-33fe8ab3fc46 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 18:59:48.921236 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f94664a-5b43-4726-b366-33fe8ab3fc46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:48.922626 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f94664a-5b43-4726-b366-33fe8ab3fc46 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:48.924741 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-1eed8b3c-ea25-4d3c-91ae-abfc9d180eaf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 02 18:59:48.924741 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 18:59:48.924741 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 18:59:48.924741 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 18:59:48.924741 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 18:59:48.924741 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 18:59:48.924741 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 18:59:48.924741 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 18:59:48.924741 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 18:59:48.924741 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 18:59:48.924741 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 18:59:48.924741 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 18:59:48.924741 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 18:59:48.924741 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 18:59:48.924741 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 18:59:48.924741 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 18:59:48.925858 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 18:59:48.925858 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 18:59:48.925858 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 18:59:48.925858 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 18:59:48.925858 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 18:59:48.925858 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 18:59:48.925858 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 18:59:48.925858 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 18:59:48.925858 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 18:59:48.925858 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 18:59:48.925858 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 18:59:48.925858 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 18:59:48.925858 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 18:59:48.925858 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 18:59:48.925858 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 18:59:48.925858 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 18:59:48.926874 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 18:59:48.926874 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 18:59:48.926874 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 18:59:48.926874 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 18:59:48.926874 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 18:59:48.926874 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 18:59:48.926874 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 18:59:48.926874 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 18:59:48.926874 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 18:59:48.926874 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 18:59:48.926874 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 02 18:59:48.926874 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 18:59:48.926874 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 95/188] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 2 18:59:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 18:59:48.926874 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eed8b3c-ea25-4d3c-91ae-abfc9d180eaf None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:48.935073 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 94/189] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 2 18:59:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:48.935929 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f94664a-5b43-4726-b366-33fe8ab3fc46 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:48.975594 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:48.988879 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:49.034736 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eed8b3c-ea25-4d3c-91ae-abfc9d180eaf None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:49.042145 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f94664a-5b43-4726-b366-33fe8ab3fc46 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:49.071826 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e966084f-f065-4381-b862-bb288096f3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:49.072511 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e966084f-f065-4381-b862-bb288096f3b0 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:49.072803 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e966084f-f065-4381-b862-bb288096f3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:49.073087 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e966084f-f065-4381-b862-bb288096f3b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:49.073858 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e966084f-f065-4381-b862-bb288096f3b0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 18:59:49.074166 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e966084f-f065-4381-b862-bb288096f3b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:49.075611 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e966084f-f065-4381-b862-bb288096f3b0 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:49.077269 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34206de0-af91-4ff6-aa83-d0e006e9accb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:49.077933 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34206de0-af91-4ff6-aa83-d0e006e9accb None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:49.078328 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34206de0-af91-4ff6-aa83-d0e006e9accb None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:49.078609 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34206de0-af91-4ff6-aa83-d0e006e9accb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:49.079645 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34206de0-af91-4ff6-aa83-d0e006e9accb None admin] RBAC: Adding query filter params (role.id=77d5301ed78146468d4115b512cc70b7, scope.project.id=7e039130ee334ac58867fed690ed99b1, user.id=720bc045e0bf473dbe5912892ff7b8e2) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 18:59:49.079964 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34206de0-af91-4ff6-aa83-d0e006e9accb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:49.081417 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34206de0-af91-4ff6-aa83-d0e006e9accb None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:49.084614 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 96/190] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 2 18:59:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:49.085447 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e966084f-f065-4381-b862-bb288096f3b0 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:49.088195 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 95/191] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 2 18:59:48 2022] GET /identity/v3/role_assignments?user.id=720bc045e0bf473dbe5912892ff7b8e2&scope.project.id=7e039130ee334ac58867fed690ed99b1&role.id=77d5301ed78146468d4115b512cc70b7 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:49.090942 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34206de0-af91-4ff6-aa83-d0e006e9accb None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:49.214834 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34206de0-af91-4ff6-aa83-d0e006e9accb None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:49.229312 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e966084f-f065-4381-b862-bb288096f3b0 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:49.229829 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:49.231339 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:49.252847 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac94792b-0e79-4a23-a67d-4f6393cf1c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:49.253767 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac94792b-0e79-4a23-a67d-4f6393cf1c9c None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:49.253988 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac94792b-0e79-4a23-a67d-4f6393cf1c9c None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:49.254269 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac94792b-0e79-4a23-a67d-4f6393cf1c9c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:49.255645 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac94792b-0e79-4a23-a67d-4f6393cf1c9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:49.257770 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac94792b-0e79-4a23-a67d-4f6393cf1c9c None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:49.259556 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 96/192] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 18:59:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:49.260411 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac94792b-0e79-4a23-a67d-4f6393cf1c9c None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:49.279363 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59a706e-bd33-43d7-a78a-02648097833c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:49.285256 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a706e-bd33-43d7-a78a-02648097833c None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:49.285256 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a706e-bd33-43d7-a78a-02648097833c None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:49.285256 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a706e-bd33-43d7-a78a-02648097833c None admin] PATH_INFO: `/v3/projects/abffdc3e7a9447068ceac783b7558c81` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:49.285256 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59a706e-bd33-43d7-a78a-02648097833c None admin] RBAC: Authorizing `identity:get_project(project_id=abffdc3e7a9447068ceac783b7558c81)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:49.285256 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59a706e-bd33-43d7-a78a-02648097833c None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:49.292177 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 97/193] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 2 18:59:49 2022] GET /identity/v3/projects/abffdc3e7a9447068ceac783b7558c81 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:49.292177 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59a706e-bd33-43d7-a78a-02648097833c None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:49.401107 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac94792b-0e79-4a23-a67d-4f6393cf1c9c None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:49.408961 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59a706e-bd33-43d7-a78a-02648097833c None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:49.438957 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072a882a-840e-4e43-b877-8e9aad860968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:49.439635 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a882a-840e-4e43-b877-8e9aad860968 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:49.439927 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a882a-840e-4e43-b877-8e9aad860968 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:49.440204 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a882a-840e-4e43-b877-8e9aad860968 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:49.440961 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072a882a-840e-4e43-b877-8e9aad860968 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 18:59:49.441270 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072a882a-840e-4e43-b877-8e9aad860968 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:49.442656 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072a882a-840e-4e43-b877-8e9aad860968 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:49.444620 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0114512-a9c2-4d88-8f6a-a756fe28e261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:49.445560 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0114512-a9c2-4d88-8f6a-a756fe28e261 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:49.445845 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0114512-a9c2-4d88-8f6a-a756fe28e261 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:49.446118 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0114512-a9c2-4d88-8f6a-a756fe28e261 None admin] PATH_INFO: `/v3/users/f830a676315f4e5c85ca846dfa43459d` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:49.452218 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 97/194] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 2 18:59:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:49.453060 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072a882a-840e-4e43-b877-8e9aad860968 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:49.461132 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0114512-a9c2-4d88-8f6a-a756fe28e261 None admin] RBAC: Authorizing `identity:get_user(user_id=f830a676315f4e5c85ca846dfa43459d)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:49.462660 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0114512-a9c2-4d88-8f6a-a756fe28e261 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:49.464437 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 98/195] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 2 18:59:49 2022] GET /identity/v3/users/f830a676315f4e5c85ca846dfa43459d => generated 283 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:49.465203 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0114512-a9c2-4d88-8f6a-a756fe28e261 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:49.494501 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:49.510223 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:49.567565 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0114512-a9c2-4d88-8f6a-a756fe28e261 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:49.570332 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072a882a-840e-4e43-b877-8e9aad860968 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:49.603139 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bb31e0-7453-4900-b062-0c57d4d24ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:49.603825 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bb31e0-7453-4900-b062-0c57d4d24ff0 None admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:49.604113 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bb31e0-7453-4900-b062-0c57d4d24ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:49.604401 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bb31e0-7453-4900-b062-0c57d4d24ff0 None admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:49.605122 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bb31e0-7453-4900-b062-0c57d4d24ff0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:49.605923 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd7b774-cd1d-4b64-8b51-0d6ceb2e2e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:49.606578 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bb31e0-7453-4900-b062-0c57d4d24ff0 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:49.607009 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd7b774-cd1d-4b64-8b51-0d6ceb2e2e36 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:49.607293 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd7b774-cd1d-4b64-8b51-0d6ceb2e2e36 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:49.607571 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd7b774-cd1d-4b64-8b51-0d6ceb2e2e36 None admin] PATH_INFO: `/v3/users/f830a676315f4e5c85ca846dfa43459d` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:49.608770 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd7b774-cd1d-4b64-8b51-0d6ceb2e2e36 None admin] RBAC: Authorizing `identity:get_user(user_id=f830a676315f4e5c85ca846dfa43459d)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:49.610212 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd7b774-cd1d-4b64-8b51-0d6ceb2e2e36 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:49.611877 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 98/196] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 2 18:59:49 2022] GET /identity/v3/users/f830a676315f4e5c85ca846dfa43459d => generated 283 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:49.617124 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd7b774-cd1d-4b64-8b51-0d6ceb2e2e36 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:49.715099 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 99/197] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 2 18:59:49 2022] POST /identity/v3/users => generated 297 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 18:59:49.715511 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd7b774-cd1d-4b64-8b51-0d6ceb2e2e36 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:49.715952 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bb31e0-7453-4900-b062-0c57d4d24ff0 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:49.741025 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:49.772456 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c4ec1b-c87b-4c29-b567-8747f6ee45bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:49.773478 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c4ec1b-c87b-4c29-b567-8747f6ee45bc None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:49.773939 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c4ec1b-c87b-4c29-b567-8747f6ee45bc None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:49.774334 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c4ec1b-c87b-4c29-b567-8747f6ee45bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:49.775927 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c4ec1b-c87b-4c29-b567-8747f6ee45bc None admin] RBAC: Adding query filter params (role.id=77d5301ed78146468d4115b512cc70b7, scope.project.id=abffdc3e7a9447068ceac783b7558c81, user.id=720bc045e0bf473dbe5912892ff7b8e2) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 18:59:49.776562 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c4ec1b-c87b-4c29-b567-8747f6ee45bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:49.781610 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c4ec1b-c87b-4c29-b567-8747f6ee45bc None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:49.796933 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 99/198] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 2 18:59:49 2022] GET /identity/v3/role_assignments?user.id=720bc045e0bf473dbe5912892ff7b8e2&scope.project.id=abffdc3e7a9447068ceac783b7558c81&role.id=77d5301ed78146468d4115b512cc70b7 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:49.796933 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c4ec1b-c87b-4c29-b567-8747f6ee45bc None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:49.847196 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:49.870940 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bb31e0-7453-4900-b062-0c57d4d24ff0 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:49.914785 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907df48a-ed6d-4071-9179-43536a5bcbaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:49.915719 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907df48a-ed6d-4071-9179-43536a5bcbaf None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:49.916003 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907df48a-ed6d-4071-9179-43536a5bcbaf None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:49.916282 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907df48a-ed6d-4071-9179-43536a5bcbaf None admin] PATH_INFO: `/v3/users/accce7a27d294457bd72f2a3cc6d8330` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:49.917705 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907df48a-ed6d-4071-9179-43536a5bcbaf None admin] RBAC: Authorizing `identity:get_user(user_id=accce7a27d294457bd72f2a3cc6d8330)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:49.919885 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907df48a-ed6d-4071-9179-43536a5bcbaf None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:49.927185 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 100/199] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 2 18:59:49 2022] GET /identity/v3/users/accce7a27d294457bd72f2a3cc6d8330 => generated 289 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:49.928315 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907df48a-ed6d-4071-9179-43536a5bcbaf None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:49.960900 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c4ec1b-c87b-4c29-b567-8747f6ee45bc None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:50.005136 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d41e0c1-c42b-4e3c-9ab4-50c2508bd997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:50.005910 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d41e0c1-c42b-4e3c-9ab4-50c2508bd997 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:50.006196 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d41e0c1-c42b-4e3c-9ab4-50c2508bd997 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:50.006477 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d41e0c1-c42b-4e3c-9ab4-50c2508bd997 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:50.067173 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907df48a-ed6d-4071-9179-43536a5bcbaf None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:50.072706 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:50.103598 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d41e0c1-c42b-4e3c-9ab4-50c2508bd997 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:50.105337 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d41e0c1-c42b-4e3c-9ab4-50c2508bd997 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:50.126213 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9337a5e4-053f-48e3-9a5a-be1b17a07598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:50.127224 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9337a5e4-053f-48e3-9a5a-be1b17a07598 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:50.127519 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9337a5e4-053f-48e3-9a5a-be1b17a07598 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:50.127803 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9337a5e4-053f-48e3-9a5a-be1b17a07598 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:50.161610 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:50.222320 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9337a5e4-053f-48e3-9a5a-be1b17a07598 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:50.224025 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9337a5e4-053f-48e3-9a5a-be1b17a07598 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:50.243562 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 100/200] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 2 18:59:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 02 18:59:50.244413 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d41e0c1-c42b-4e3c-9ab4-50c2508bd997 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:50.329925 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:50.335361 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 101/201] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 2 18:59:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 02 18:59:50.336411 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9337a5e4-053f-48e3-9a5a-be1b17a07598 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:50.359066 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d41e0c1-c42b-4e3c-9ab4-50c2508bd997 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:50.395161 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:50.400248 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ce7518-4912-414b-b2a8-b6d72d5b42d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:50.401063 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ce7518-4912-414b-b2a8-b6d72d5b42d4 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:50.401348 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ce7518-4912-414b-b2a8-b6d72d5b42d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:50.401623 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ce7518-4912-414b-b2a8-b6d72d5b42d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:50.470778 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9337a5e4-053f-48e3-9a5a-be1b17a07598 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:50.498512 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ce7518-4912-414b-b2a8-b6d72d5b42d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:50.500044 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ce7518-4912-414b-b2a8-b6d72d5b42d4 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:50.501348 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b182cbb-ca14-4ac1-bfaf-660fce631945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:50.502492 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b182cbb-ca14-4ac1-bfaf-660fce631945 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:50.502804 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b182cbb-ca14-4ac1-bfaf-660fce631945 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:50.503085 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b182cbb-ca14-4ac1-bfaf-660fce631945 None admin] PATH_INFO: `/v3/projects/5f19550a1b254991ba8eb40da4841fcf` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:50.504528 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b182cbb-ca14-4ac1-bfaf-660fce631945 None admin] RBAC: Authorizing `identity:get_project(project_id=5f19550a1b254991ba8eb40da4841fcf)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:50.506600 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b182cbb-ca14-4ac1-bfaf-660fce631945 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:50.508731 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 102/202] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 2 18:59:50 2022] GET /identity/v3/projects/5f19550a1b254991ba8eb40da4841fcf => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:50.509526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b182cbb-ca14-4ac1-bfaf-660fce631945 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:50.606619 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:50.634707 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b182cbb-ca14-4ac1-bfaf-660fce631945 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:50.635635 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 101/203] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 2 18:59:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 02 18:59:50.636587 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ce7518-4912-414b-b2a8-b6d72d5b42d4 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:50.661591 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:50.675992 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9982c6-7175-4fa2-bf8e-b917068cea12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:50.676878 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9982c6-7175-4fa2-bf8e-b917068cea12 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:50.677159 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9982c6-7175-4fa2-bf8e-b917068cea12 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:50.677432 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9982c6-7175-4fa2-bf8e-b917068cea12 None admin] PATH_INFO: `/v3/projects/5f19550a1b254991ba8eb40da4841fcf` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:50.678672 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9982c6-7175-4fa2-bf8e-b917068cea12 None admin] RBAC: Authorizing `identity:get_project(project_id=5f19550a1b254991ba8eb40da4841fcf)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:50.680309 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9982c6-7175-4fa2-bf8e-b917068cea12 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:50.682391 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 103/204] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 2 18:59:50 2022] GET /identity/v3/projects/5f19550a1b254991ba8eb40da4841fcf => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:50.683316 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9982c6-7175-4fa2-bf8e-b917068cea12 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:50.750419 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ce7518-4912-414b-b2a8-b6d72d5b42d4 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:50.790073 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632b0fbf-1c7e-42f7-a782-98204bd005b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:50.790960 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b0fbf-1c7e-42f7-a782-98204bd005b3 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:50.791254 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b0fbf-1c7e-42f7-a782-98204bd005b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:50.791850 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b0fbf-1c7e-42f7-a782-98204bd005b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:50.793020 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632b0fbf-1c7e-42f7-a782-98204bd005b3 None admin] RBAC: Adding query filter params (role.id=2e0c054b646e4ed2a0e7e5962f09a50e, scope.project.id=5f19550a1b254991ba8eb40da4841fcf, user.id=f830a676315f4e5c85ca846dfa43459d) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 18:59:50.793340 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632b0fbf-1c7e-42f7-a782-98204bd005b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:50.794407 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9982c6-7175-4fa2-bf8e-b917068cea12 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:50.794947 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632b0fbf-1c7e-42f7-a782-98204bd005b3 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:50.806860 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 102/205] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 2 18:59:50 2022] GET /identity/v3/role_assignments?user.id=f830a676315f4e5c85ca846dfa43459d&scope.project.id=5f19550a1b254991ba8eb40da4841fcf&role.id=2e0c054b646e4ed2a0e7e5962f09a50e => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:50.807911 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632b0fbf-1c7e-42f7-a782-98204bd005b3 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:50.861312 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6577c67c-8597-4e19-b835-d9b2b99bfee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:50.862120 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6577c67c-8597-4e19-b835-d9b2b99bfee1 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:50.862400 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6577c67c-8597-4e19-b835-d9b2b99bfee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:50.862674 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6577c67c-8597-4e19-b835-d9b2b99bfee1 None admin] PATH_INFO: `/v3/projects/abffdc3e7a9447068ceac783b7558c81` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:50.863991 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6577c67c-8597-4e19-b835-d9b2b99bfee1 None admin] RBAC: Authorizing `identity:get_project(project_id=abffdc3e7a9447068ceac783b7558c81)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:50.865592 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6577c67c-8597-4e19-b835-d9b2b99bfee1 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:50.867384 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 104/206] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 2 18:59:50 2022] GET /identity/v3/projects/abffdc3e7a9447068ceac783b7558c81 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:50.873418 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6577c67c-8597-4e19-b835-d9b2b99bfee1 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:50.908143 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:50.985684 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:51.048557 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632b0fbf-1c7e-42f7-a782-98204bd005b3 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:51.071728 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6577c67c-8597-4e19-b835-d9b2b99bfee1 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:51.092314 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b2a9d-1aee-480a-8821-a37584e6fc9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:51.092982 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b2a9d-1aee-480a-8821-a37584e6fc9c None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:51.093262 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b2a9d-1aee-480a-8821-a37584e6fc9c None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:51.093994 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b2a9d-1aee-480a-8821-a37584e6fc9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:51.095126 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6b2a9d-1aee-480a-8821-a37584e6fc9c None admin] RBAC: Adding query filter params (role.id=f13fdd0847924014abf1e95594195e78, scope.project.id=5f19550a1b254991ba8eb40da4841fcf, user.id=f830a676315f4e5c85ca846dfa43459d) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 18:59:51.095458 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6b2a9d-1aee-480a-8821-a37584e6fc9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:51.097385 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6b2a9d-1aee-480a-8821-a37584e6fc9c None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:51.105390 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 103/207] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 2 18:59:50 2022] GET /identity/v3/role_assignments?user.id=f830a676315f4e5c85ca846dfa43459d&scope.project.id=5f19550a1b254991ba8eb40da4841fcf&role.id=f13fdd0847924014abf1e95594195e78 => generated 259 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:51.139697 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1318803-943b-41e8-b580-af83b072bc00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:51.140402 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1318803-943b-41e8-b580-af83b072bc00 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:51.140726 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1318803-943b-41e8-b580-af83b072bc00 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:51.141004 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1318803-943b-41e8-b580-af83b072bc00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:51.142152 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1318803-943b-41e8-b580-af83b072bc00 None admin] RBAC: Adding query filter params (role.id=de9eea600632474d8b149397589c3746, scope.project.id=abffdc3e7a9447068ceac783b7558c81, user.id=accce7a27d294457bd72f2a3cc6d8330) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 18:59:51.142469 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1318803-943b-41e8-b580-af83b072bc00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:51.144084 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1318803-943b-41e8-b580-af83b072bc00 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:51.161271 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 105/208] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 2 18:59:50 2022] GET /identity/v3/role_assignments?user.id=accce7a27d294457bd72f2a3cc6d8330&scope.project.id=abffdc3e7a9447068ceac783b7558c81&role.id=de9eea600632474d8b149397589c3746 => generated 608 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 18:59:52.357042 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78745f6d-32d0-471c-a2e9-a240ec2cedc1 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:52.357727 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78745f6d-32d0-471c-a2e9-a240ec2cedc1 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:52.357727 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78745f6d-32d0-471c-a2e9-a240ec2cedc1 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:52.359238 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 104/209] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 18:59:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 18:59:52.364571 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba75242-21a1-4698-a81e-4e8c4cd91861 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:52.365099 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba75242-21a1-4698-a81e-4e8c4cd91861 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:52.365099 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba75242-21a1-4698-a81e-4e8c4cd91861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:52.370062 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d175fc3a-b221-486c-b02f-eb6cb1a71461 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:52.370277 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d175fc3a-b221-486c-b02f-eb6cb1a71461 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:52.370547 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d175fc3a-b221-486c-b02f-eb6cb1a71461 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:52.373161 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 105/210] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 18:59:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 02 18:59:52.379241 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d67cd0-ec51-47c4-a3ec-6662fccc248b None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:52.379528 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d67cd0-ec51-47c4-a3ec-6662fccc248b None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:52.379804 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d67cd0-ec51-47c4-a3ec-6662fccc248b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:52.383964 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-5ba75242-21a1-4698-a81e-4e8c4cd91861 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 18:59:52.385249 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-5ba75242-21a1-4698-a81e-4e8c4cd91861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 18:59:52.400453 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-80d67cd0-ec51-47c4-a3ec-6662fccc248b None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 18:59:52.401934 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-80d67cd0-ec51-47c4-a3ec-6662fccc248b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 18:59:52.472025 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 106/211] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 18:59:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 18:59:52.480369 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 106/212] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 18:59:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 02 18:59:52.497500 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba75242-21a1-4698-a81e-4e8c4cd91861 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:52.506429 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d67cd0-ec51-47c4-a3ec-6662fccc248b None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:52.541515 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:52.570552 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:52.618870 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba75242-21a1-4698-a81e-4e8c4cd91861 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:52.627684 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d67cd0-ec51-47c4-a3ec-6662fccc248b None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:52.660738 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcfe827-2a6c-4b09-91c9-4f42ce88a2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:52.661971 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcfe827-2a6c-4b09-91c9-4f42ce88a2b9 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:52.662937 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcfe827-2a6c-4b09-91c9-4f42ce88a2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:52.663310 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcfe827-2a6c-4b09-91c9-4f42ce88a2b9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:52.665258 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcfe827-2a6c-4b09-91c9-4f42ce88a2b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:52.667410 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcfe827-2a6c-4b09-91c9-4f42ce88a2b9 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:52.669650 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 107/213] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 18:59:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:52.674069 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53060bd-3eae-4823-8222-673a273d02cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:52.674069 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53060bd-3eae-4823-8222-673a273d02cc None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:52.674069 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53060bd-3eae-4823-8222-673a273d02cc None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:52.674069 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53060bd-3eae-4823-8222-673a273d02cc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:52.675753 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcfe827-2a6c-4b09-91c9-4f42ce88a2b9 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:52.686930 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53060bd-3eae-4823-8222-673a273d02cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 18:59:52.695610 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-b53060bd-3eae-4823-8222-673a273d02cc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 02 18:59:52.695610 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 18:59:52.695610 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 18:59:52.695610 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 18:59:52.695610 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 18:59:52.695610 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 18:59:52.695610 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 18:59:52.695610 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 18:59:52.695610 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 18:59:52.695610 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 18:59:52.695610 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 18:59:52.695610 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 18:59:52.695610 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 18:59:52.695610 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 18:59:52.695610 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 18:59:52.695610 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 18:59:52.696533 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 18:59:52.696533 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 18:59:52.696533 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 18:59:52.696533 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 18:59:52.696533 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 18:59:52.696533 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 18:59:52.696533 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 18:59:52.696533 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 18:59:52.696533 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 18:59:52.696533 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 18:59:52.696533 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 18:59:52.696533 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 18:59:52.696533 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 18:59:52.696533 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 18:59:52.696533 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 18:59:52.696533 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 18:59:52.697621 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 18:59:52.697621 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 18:59:52.697621 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 18:59:52.697621 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 18:59:52.697621 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 18:59:52.697621 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 18:59:52.697621 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 18:59:52.697621 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 18:59:52.697621 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 18:59:52.697621 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 18:59:52.697621 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 02 18:59:52.697621 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 18:59:52.702308 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 107/214] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 2 18:59:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 02 18:59:52.703081 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53060bd-3eae-4823-8222-673a273d02cc None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:52.826715 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:52.826923 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcfe827-2a6c-4b09-91c9-4f42ce88a2b9 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:52.836778 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53060bd-3eae-4823-8222-673a273d02cc None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:52.846035 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:52.864128 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db17efe-be70-4732-8f00-b3dc06255465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:52.864910 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db17efe-be70-4732-8f00-b3dc06255465 None admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:52.865348 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db17efe-be70-4732-8f00-b3dc06255465 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:52.865689 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db17efe-be70-4732-8f00-b3dc06255465 None admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:52.866504 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db17efe-be70-4732-8f00-b3dc06255465 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:52.868139 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db17efe-be70-4732-8f00-b3dc06255465 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:52.886423 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f20dbbc-f6ef-40bd-94b4-33738406201d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:52.887162 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f20dbbc-f6ef-40bd-94b4-33738406201d None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:52.887516 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f20dbbc-f6ef-40bd-94b4-33738406201d None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:52.887861 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f20dbbc-f6ef-40bd-94b4-33738406201d None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:52.888851 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f20dbbc-f6ef-40bd-94b4-33738406201d None admin] RBAC: Adding query filter params (name=member) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 18:59:52.889282 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f20dbbc-f6ef-40bd-94b4-33738406201d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:52.890799 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f20dbbc-f6ef-40bd-94b4-33738406201d None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:52.901610 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 108/215] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 2 18:59:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:52.907036 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f20dbbc-f6ef-40bd-94b4-33738406201d None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:53.009472 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 108/216] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 2 18:59:52 2022] POST /identity/v3/users => generated 297 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 02 18:59:53.036135 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f20dbbc-f6ef-40bd-94b4-33738406201d None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:53.082076 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a4fb30-9113-4020-b06b-134568e70049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:53.083008 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a4fb30-9113-4020-b06b-134568e70049 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:53.083315 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a4fb30-9113-4020-b06b-134568e70049 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:53.083595 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a4fb30-9113-4020-b06b-134568e70049 None admin] PATH_INFO: `/v3/users/7aae3b53a99b47378ed66d7870667d89` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:53.117465 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a4fb30-9113-4020-b06b-134568e70049 None admin] RBAC: Authorizing `identity:get_user(user_id=7aae3b53a99b47378ed66d7870667d89)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:53.119058 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a4fb30-9113-4020-b06b-134568e70049 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:53.121111 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 109/217] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 2 18:59:52 2022] GET /identity/v3/users/7aae3b53a99b47378ed66d7870667d89 => generated 297 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 18:59:53.128902 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db17efe-be70-4732-8f00-b3dc06255465 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:53.277230 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db17efe-be70-4732-8f00-b3dc06255465 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:53.306385 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef19b949-291d-42d4-8bdf-c5ef607febdd None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:53.306722 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef19b949-291d-42d4-8bdf-c5ef607febdd None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:53.306979 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef19b949-291d-42d4-8bdf-c5ef607febdd None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:53.308683 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 110/218] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 18:59:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 02 18:59:53.314557 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae09615b-c862-4cbf-9b30-4b9c654ba0c0 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:53.314883 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae09615b-c862-4cbf-9b30-4b9c654ba0c0 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:53.315163 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae09615b-c862-4cbf-9b30-4b9c654ba0c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:53.322225 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511b847c-5d2f-4716-ba5a-714af8bb8a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:53.322991 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511b847c-5d2f-4716-ba5a-714af8bb8a04 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:53.323285 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511b847c-5d2f-4716-ba5a-714af8bb8a04 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:53.323562 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511b847c-5d2f-4716-ba5a-714af8bb8a04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:53.343171 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-ae09615b-c862-4cbf-9b30-4b9c654ba0c0 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 18:59:53.345721 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-ae09615b-c862-4cbf-9b30-4b9c654ba0c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 18:59:53.363698 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:53.378237 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:53.406805 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511b847c-5d2f-4716-ba5a-714af8bb8a04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:53.408542 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511b847c-5d2f-4716-ba5a-714af8bb8a04 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:53.472038 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 111/219] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 18:59:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 18:59:53.505579 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae09615b-c862-4cbf-9b30-4b9c654ba0c0 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:53.535556 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 109/220] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 2 18:59:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 02 18:59:53.541216 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511b847c-5d2f-4716-ba5a-714af8bb8a04 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:53.640645 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:53.686868 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511b847c-5d2f-4716-ba5a-714af8bb8a04 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:53.710192 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae09615b-c862-4cbf-9b30-4b9c654ba0c0 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:53.729353 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6895f871-22d5-4d53-a39b-3523b845089c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:53.730320 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6895f871-22d5-4d53-a39b-3523b845089c None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:53.730667 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6895f871-22d5-4d53-a39b-3523b845089c None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:53.731038 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6895f871-22d5-4d53-a39b-3523b845089c None admin] PATH_INFO: `/v3/projects/5f19550a1b254991ba8eb40da4841fcf` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:53.732341 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6895f871-22d5-4d53-a39b-3523b845089c None admin] RBAC: Authorizing `identity:get_project(project_id=5f19550a1b254991ba8eb40da4841fcf)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:53.733848 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6895f871-22d5-4d53-a39b-3523b845089c None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:53.736057 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 110/221] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 2 18:59:53 2022] GET /identity/v3/projects/5f19550a1b254991ba8eb40da4841fcf => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:53.746371 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6895f871-22d5-4d53-a39b-3523b845089c None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:53.753725 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cc9473-db49-41dd-8be0-2df5e3e1ba55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:53.754621 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cc9473-db49-41dd-8be0-2df5e3e1ba55 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:53.754929 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cc9473-db49-41dd-8be0-2df5e3e1ba55 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:53.755258 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cc9473-db49-41dd-8be0-2df5e3e1ba55 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:53.766834 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cc9473-db49-41dd-8be0-2df5e3e1ba55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 18:59:53.774992 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-82cc9473-db49-41dd-8be0-2df5e3e1ba55 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 02 18:59:53.774992 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 18:59:53.774992 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 18:59:53.774992 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 18:59:53.774992 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 18:59:53.774992 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 18:59:53.774992 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 18:59:53.774992 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 18:59:53.774992 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 18:59:53.774992 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 18:59:53.774992 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 18:59:53.774992 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 18:59:53.774992 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 18:59:53.774992 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 18:59:53.774992 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 18:59:53.774992 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 18:59:53.775882 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 18:59:53.775882 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 18:59:53.775882 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 18:59:53.775882 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 18:59:53.775882 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 18:59:53.775882 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 18:59:53.775882 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 18:59:53.775882 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 18:59:53.775882 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 18:59:53.775882 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 18:59:53.775882 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 18:59:53.775882 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 18:59:53.775882 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 18:59:53.775882 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 18:59:53.775882 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 18:59:53.775882 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 18:59:53.776942 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 18:59:53.776942 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 18:59:53.776942 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 18:59:53.776942 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 18:59:53.776942 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 18:59:53.776942 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 18:59:53.776942 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 18:59:53.776942 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 18:59:53.776942 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 18:59:53.776942 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 18:59:53.776942 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 02 18:59:53.776942 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 18:59:53.778304 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 112/222] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 2 18:59:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 18:59:53.779401 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28677ffc-4e0f-40f5-9e5e-08c5ccda490d None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:53.779901 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28677ffc-4e0f-40f5-9e5e-08c5ccda490d None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:53.780393 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28677ffc-4e0f-40f5-9e5e-08c5ccda490d None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:53.781993 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 113/223] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 18:59:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 18:59:53.782944 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28677ffc-4e0f-40f5-9e5e-08c5ccda490d None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:53.787768 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:53.941771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6895f871-22d5-4d53-a39b-3523b845089c None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:53.978473 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf365efa-c24a-479c-94c6-40c9ce9e641d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:53.979278 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf365efa-c24a-479c-94c6-40c9ce9e641d None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:53.979588 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf365efa-c24a-479c-94c6-40c9ce9e641d None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:53.979864 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf365efa-c24a-479c-94c6-40c9ce9e641d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:53.980707 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf365efa-c24a-479c-94c6-40c9ce9e641d None admin] RBAC: Adding query filter params (role.id=77d5301ed78146468d4115b512cc70b7, scope.project.id=5f19550a1b254991ba8eb40da4841fcf, user.id=7aae3b53a99b47378ed66d7870667d89) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 18:59:53.981063 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf365efa-c24a-479c-94c6-40c9ce9e641d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:53.982496 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf365efa-c24a-479c-94c6-40c9ce9e641d None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:53.987610 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28677ffc-4e0f-40f5-9e5e-08c5ccda490d None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:53.988582 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 111/224] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 2 18:59:53 2022] GET /identity/v3/role_assignments?user.id=7aae3b53a99b47378ed66d7870667d89&scope.project.id=5f19550a1b254991ba8eb40da4841fcf&role.id=77d5301ed78146468d4115b512cc70b7 => generated 259 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:53.989842 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cdef43-aa62-4f46-bcef-6af858294d0f None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:53.990128 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cdef43-aa62-4f46-bcef-6af858294d0f None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:53.991150 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cdef43-aa62-4f46-bcef-6af858294d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:54.005246 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:54.020155 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-45cdef43-aa62-4f46-bcef-6af858294d0f None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 18:59:54.021404 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-45cdef43-aa62-4f46-bcef-6af858294d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 18:59:54.046304 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e92c058-a6bb-4097-b2dd-fc6b3205299c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:54.046947 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e92c058-a6bb-4097-b2dd-fc6b3205299c None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:54.047232 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e92c058-a6bb-4097-b2dd-fc6b3205299c None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:54.048276 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e92c058-a6bb-4097-b2dd-fc6b3205299c None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:54.049061 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e92c058-a6bb-4097-b2dd-fc6b3205299c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 18:59:54.049372 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e92c058-a6bb-4097-b2dd-fc6b3205299c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:54.054011 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e92c058-a6bb-4097-b2dd-fc6b3205299c None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:54.071994 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 114/225] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 2 18:59:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:54.074160 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e92c058-a6bb-4097-b2dd-fc6b3205299c None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:54.175167 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:54.197832 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 112/226] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 18:59:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 18:59:54.227969 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cdef43-aa62-4f46-bcef-6af858294d0f None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:54.230044 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e92c058-a6bb-4097-b2dd-fc6b3205299c None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:54.327566 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb91bfaa-1a7c-4722-b863-84bda4f6342e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:54.328396 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb91bfaa-1a7c-4722-b863-84bda4f6342e None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:54.328685 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb91bfaa-1a7c-4722-b863-84bda4f6342e None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:54.328967 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb91bfaa-1a7c-4722-b863-84bda4f6342e None admin] PATH_INFO: `/v3/users/f830a676315f4e5c85ca846dfa43459d` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:54.335541 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb91bfaa-1a7c-4722-b863-84bda4f6342e None admin] RBAC: Authorizing `identity:get_user(user_id=f830a676315f4e5c85ca846dfa43459d)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:54.337047 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb91bfaa-1a7c-4722-b863-84bda4f6342e None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:54.342925 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 115/227] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 2 18:59:54 2022] GET /identity/v3/users/f830a676315f4e5c85ca846dfa43459d => generated 283 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:54.345181 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cdef43-aa62-4f46-bcef-6af858294d0f None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:54.360483 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb91bfaa-1a7c-4722-b863-84bda4f6342e None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:54.386557 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36a20df-5c8e-4436-8de9-b4542ca4c4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:54.388029 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36a20df-5c8e-4436-8de9-b4542ca4c4d5 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:54.388460 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36a20df-5c8e-4436-8de9-b4542ca4c4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:54.388948 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36a20df-5c8e-4436-8de9-b4542ca4c4d5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:54.398084 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:54.403400 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36a20df-5c8e-4436-8de9-b4542ca4c4d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 18:59:54.412558 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-e36a20df-5c8e-4436-8de9-b4542ca4c4d5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 02 18:59:54.412558 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 18:59:54.412558 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 18:59:54.412558 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 18:59:54.412558 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 18:59:54.412558 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 18:59:54.412558 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 18:59:54.412558 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 18:59:54.412558 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 18:59:54.412558 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 18:59:54.412558 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 18:59:54.412558 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 18:59:54.412558 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 18:59:54.412558 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 18:59:54.412558 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 18:59:54.412558 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 18:59:54.413479 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 18:59:54.413479 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 18:59:54.413479 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 18:59:54.413479 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 18:59:54.413479 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 18:59:54.413479 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 18:59:54.413479 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 18:59:54.413479 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 18:59:54.413479 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 18:59:54.413479 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 18:59:54.413479 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 18:59:54.413479 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 18:59:54.413479 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 18:59:54.413479 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 18:59:54.413479 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 18:59:54.413479 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 18:59:54.414285 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 18:59:54.414285 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 18:59:54.414285 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 18:59:54.414285 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 18:59:54.414285 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 18:59:54.414285 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 18:59:54.414285 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 18:59:54.414285 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 18:59:54.414285 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 18:59:54.414285 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 18:59:54.414285 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 02 18:59:54.414285 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 18:59:54.417146 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 113/228] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 2 18:59:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 02 18:59:54.423100 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36a20df-5c8e-4436-8de9-b4542ca4c4d5 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:54.598720 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb91bfaa-1a7c-4722-b863-84bda4f6342e None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:54.644548 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e99b795-8d1f-442c-9fa5-9d7db7ee1b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:54.646038 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e99b795-8d1f-442c-9fa5-9d7db7ee1b13 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:54.646538 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e99b795-8d1f-442c-9fa5-9d7db7ee1b13 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:54.647837 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e99b795-8d1f-442c-9fa5-9d7db7ee1b13 None admin] PATH_INFO: `/v3/projects/5f19550a1b254991ba8eb40da4841fcf` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:54.649576 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e99b795-8d1f-442c-9fa5-9d7db7ee1b13 None admin] RBAC: Authorizing `identity:get_project(project_id=5f19550a1b254991ba8eb40da4841fcf)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:54.651824 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e99b795-8d1f-442c-9fa5-9d7db7ee1b13 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:54.653893 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 116/229] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 2 18:59:54 2022] GET /identity/v3/projects/5f19550a1b254991ba8eb40da4841fcf => generated 302 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:54.660573 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e99b795-8d1f-442c-9fa5-9d7db7ee1b13 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:54.661557 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36a20df-5c8e-4436-8de9-b4542ca4c4d5 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:54.672018 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:54.752682 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108b1cf2-b111-4596-9404-559886f5ac8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:54.753835 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b1cf2-b111-4596-9404-559886f5ac8c None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:54.754684 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b1cf2-b111-4596-9404-559886f5ac8c None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:54.755236 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b1cf2-b111-4596-9404-559886f5ac8c None admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:54.756303 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108b1cf2-b111-4596-9404-559886f5ac8c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 18:59:54.756830 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108b1cf2-b111-4596-9404-559886f5ac8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:54.758711 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108b1cf2-b111-4596-9404-559886f5ac8c None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:54.780828 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 114/230] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 2 18:59:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:54.786833 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108b1cf2-b111-4596-9404-559886f5ac8c None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:54.796750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e99b795-8d1f-442c-9fa5-9d7db7ee1b13 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:54.846779 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb0f178-1d97-45b0-9c07-375ad4b244d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:54.848326 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb0f178-1d97-45b0-9c07-375ad4b244d4 None admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:54.848620 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb0f178-1d97-45b0-9c07-375ad4b244d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:54.848996 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb0f178-1d97-45b0-9c07-375ad4b244d4 None admin] PATH_INFO: `/v3/projects/5f19550a1b254991ba8eb40da4841fcf/users/f830a676315f4e5c85ca846dfa43459d/roles/2e0c054b646e4ed2a0e7e5962f09a50e` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:54.858109 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb0f178-1d97-45b0-9c07-375ad4b244d4 None admin] RBAC: Authorizing `identity:create_grant(project_id=5f19550a1b254991ba8eb40da4841fcf, user_id=f830a676315f4e5c85ca846dfa43459d, role_id=2e0c054b646e4ed2a0e7e5962f09a50e)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:54.860311 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb0f178-1d97-45b0-9c07-375ad4b244d4 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:54.919252 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:54.933565 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 117/231] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 2 18:59:54 2022] PUT /identity/v3/projects/5f19550a1b254991ba8eb40da4841fcf/users/f830a676315f4e5c85ca846dfa43459d/roles/2e0c054b646e4ed2a0e7e5962f09a50e => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 02 18:59:54.963627 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108b1cf2-b111-4596-9404-559886f5ac8c None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:55.012607 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25d6137-702f-4fc3-97ed-e93aee3e50ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:55.013541 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d6137-702f-4fc3-97ed-e93aee3e50ac None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:55.013824 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d6137-702f-4fc3-97ed-e93aee3e50ac None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:55.014096 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d6137-702f-4fc3-97ed-e93aee3e50ac None admin] PATH_INFO: `/v3/users/f830a676315f4e5c85ca846dfa43459d` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:55.015629 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25d6137-702f-4fc3-97ed-e93aee3e50ac None admin] RBAC: Authorizing `identity:get_user(user_id=f830a676315f4e5c85ca846dfa43459d)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:55.017280 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25d6137-702f-4fc3-97ed-e93aee3e50ac None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:55.019373 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 115/232] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 2 18:59:54 2022] GET /identity/v3/users/f830a676315f4e5c85ca846dfa43459d => generated 283 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 18:59:55.028065 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb0f178-1d97-45b0-9c07-375ad4b244d4 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:55.170962 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:55.222971 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb0f178-1d97-45b0-9c07-375ad4b244d4 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:55.266134 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95822da2-975b-48d4-9fc6-8c8fd99bf128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:55.267057 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95822da2-975b-48d4-9fc6-8c8fd99bf128 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:55.267344 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95822da2-975b-48d4-9fc6-8c8fd99bf128 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:55.267621 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95822da2-975b-48d4-9fc6-8c8fd99bf128 None admin] PATH_INFO: `/v3/projects/5f19550a1b254991ba8eb40da4841fcf` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:55.268921 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95822da2-975b-48d4-9fc6-8c8fd99bf128 None admin] RBAC: Authorizing `identity:get_project(project_id=5f19550a1b254991ba8eb40da4841fcf)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:55.270432 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95822da2-975b-48d4-9fc6-8c8fd99bf128 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:55.277391 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 118/233] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 2 18:59:55 2022] GET /identity/v3/projects/5f19550a1b254991ba8eb40da4841fcf => generated 302 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:55.278782 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25d6137-702f-4fc3-97ed-e93aee3e50ac None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:55.426409 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25d6137-702f-4fc3-97ed-e93aee3e50ac None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:55.497640 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:55.519682 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5349fd8-f382-4b90-9bea-7b3f1f5ccecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:55.521250 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5349fd8-f382-4b90-9bea-7b3f1f5ccecb None admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:55.522248 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5349fd8-f382-4b90-9bea-7b3f1f5ccecb None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:55.522720 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5349fd8-f382-4b90-9bea-7b3f1f5ccecb None admin] PATH_INFO: `/v3/projects/5f19550a1b254991ba8eb40da4841fcf/users/f830a676315f4e5c85ca846dfa43459d/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:55.529314 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5349fd8-f382-4b90-9bea-7b3f1f5ccecb None admin] RBAC: Authorizing `identity:create_grant(project_id=5f19550a1b254991ba8eb40da4841fcf, user_id=f830a676315f4e5c85ca846dfa43459d, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:55.532086 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5349fd8-f382-4b90-9bea-7b3f1f5ccecb None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:55.651273 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 116/234] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 2 18:59:55 2022] PUT /identity/v3/projects/5f19550a1b254991ba8eb40da4841fcf/users/f830a676315f4e5c85ca846dfa43459d/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 18:59:56.071055 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5767be11-0ab1-4dae-bd2f-d78ed43b5e99 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:56.071599 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5767be11-0ab1-4dae-bd2f-d78ed43b5e99 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:56.071599 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5767be11-0ab1-4dae-bd2f-d78ed43b5e99 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:56.073004 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 119/235] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 18:59:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 18:59:56.078564 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1404566e-5d42-4457-a8c1-53648f8f91d8 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:56.078890 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1404566e-5d42-4457-a8c1-53648f8f91d8 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:56.079175 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1404566e-5d42-4457-a8c1-53648f8f91d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:56.103797 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-1404566e-5d42-4457-a8c1-53648f8f91d8 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 18:59:56.105019 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-1404566e-5d42-4457-a8c1-53648f8f91d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 18:59:56.190888 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bc0408-fc4c-4543-8f24-4b1703fb8816 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:56.191211 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bc0408-fc4c-4543-8f24-4b1703fb8816 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:56.191523 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bc0408-fc4c-4543-8f24-4b1703fb8816 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:56.193165 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 120/236] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 18:59:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 18:59:56.199463 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec5a133-ead5-4f78-934b-f5b265caec94 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:56.199760 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec5a133-ead5-4f78-934b-f5b265caec94 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:56.200046 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec5a133-ead5-4f78-934b-f5b265caec94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:56.208311 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 117/237] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 18:59:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 18:59:56.226838 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-fec5a133-ead5-4f78-934b-f5b265caec94 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 18:59:56.228256 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-fec5a133-ead5-4f78-934b-f5b265caec94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 18:59:56.248403 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1404566e-5d42-4457-a8c1-53648f8f91d8 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:56.354529 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 121/238] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 18:59:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 02 18:59:56.395828 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec5a133-ead5-4f78-934b-f5b265caec94 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:56.412157 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1404566e-5d42-4457-a8c1-53648f8f91d8 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:56.432183 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:56.456128 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80d6705-cda6-4f66-8423-7033615c1430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:56.457072 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80d6705-cda6-4f66-8423-7033615c1430 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:56.457358 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80d6705-cda6-4f66-8423-7033615c1430 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:56.457634 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80d6705-cda6-4f66-8423-7033615c1430 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:56.491945 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80d6705-cda6-4f66-8423-7033615c1430 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 18:59:56.500011 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-f80d6705-cda6-4f66-8423-7033615c1430 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 02 18:59:56.500011 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 18:59:56.500011 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 18:59:56.500011 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 18:59:56.500011 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 18:59:56.500011 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 18:59:56.500011 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 18:59:56.500011 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 18:59:56.500011 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 18:59:56.500011 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 18:59:56.500011 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 18:59:56.500011 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 18:59:56.500011 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 18:59:56.500011 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 18:59:56.500011 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 18:59:56.500011 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 18:59:56.507815 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 18:59:56.507815 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 18:59:56.507815 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 18:59:56.507815 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 18:59:56.507815 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 18:59:56.507815 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 18:59:56.507815 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 18:59:56.507815 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 18:59:56.507815 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 18:59:56.507815 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 18:59:56.507815 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 18:59:56.507815 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 18:59:56.507815 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 18:59:56.507815 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 18:59:56.507815 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 18:59:56.507815 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 18:59:56.509316 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 18:59:56.509316 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 18:59:56.509316 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 18:59:56.509316 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 18:59:56.509316 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 18:59:56.509316 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 18:59:56.509316 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 18:59:56.509316 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 18:59:56.509316 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 18:59:56.509316 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 18:59:56.509316 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 02 18:59:56.509316 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 18:59:56.509316 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 118/239] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 2 18:59:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 02 18:59:56.509316 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80d6705-cda6-4f66-8423-7033615c1430 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:56.527410 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:56.568209 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec5a133-ead5-4f78-934b-f5b265caec94 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:56.609969 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aa2f48-4f7a-48d6-bc1e-5d598848c1b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:56.611285 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa2f48-4f7a-48d6-bc1e-5d598848c1b3 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:56.611590 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa2f48-4f7a-48d6-bc1e-5d598848c1b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:56.611871 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa2f48-4f7a-48d6-bc1e-5d598848c1b3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:56.613141 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aa2f48-4f7a-48d6-bc1e-5d598848c1b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:56.614882 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aa2f48-4f7a-48d6-bc1e-5d598848c1b3 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:56.616619 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 122/240] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 18:59:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:56.622632 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aa2f48-4f7a-48d6-bc1e-5d598848c1b3 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:56.738400 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aa2f48-4f7a-48d6-bc1e-5d598848c1b3 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:56.773910 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:56.783539 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcada59c-9f95-4aa3-84ca-c9449cac00a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:56.784291 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcada59c-9f95-4aa3-84ca-c9449cac00a1 None admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:56.784831 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcada59c-9f95-4aa3-84ca-c9449cac00a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:56.785176 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcada59c-9f95-4aa3-84ca-c9449cac00a1 None admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:56.785984 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcada59c-9f95-4aa3-84ca-c9449cac00a1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:56.788201 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcada59c-9f95-4aa3-84ca-c9449cac00a1 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:56.820696 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80d6705-cda6-4f66-8423-7033615c1430 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:56.861017 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bff98c-c33f-4348-8a7a-a1a4f265fa34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:56.861830 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bff98c-c33f-4348-8a7a-a1a4f265fa34 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:56.862268 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bff98c-c33f-4348-8a7a-a1a4f265fa34 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:56.862734 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bff98c-c33f-4348-8a7a-a1a4f265fa34 None admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:56.864118 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bff98c-c33f-4348-8a7a-a1a4f265fa34 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 18:59:56.864554 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bff98c-c33f-4348-8a7a-a1a4f265fa34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:56.867344 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bff98c-c33f-4348-8a7a-a1a4f265fa34 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:56.880980 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 119/241] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 2 18:59:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:56.889073 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bff98c-c33f-4348-8a7a-a1a4f265fa34 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:56.891588 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 123/242] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 2 18:59:56 2022] POST /identity/v3/users => generated 293 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 18:59:56.968809 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:57.066612 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bff98c-c33f-4348-8a7a-a1a4f265fa34 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:57.121128 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9072f5-9eb9-491d-aa4f-406ff1ee8275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:57.122277 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9072f5-9eb9-491d-aa4f-406ff1ee8275 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:57.122559 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9072f5-9eb9-491d-aa4f-406ff1ee8275 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:57.122911 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9072f5-9eb9-491d-aa4f-406ff1ee8275 None admin] PATH_INFO: `/v3/users/dffa20f30b714c7580ce966ceba58add` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:57.143108 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9072f5-9eb9-491d-aa4f-406ff1ee8275 None admin] RBAC: Authorizing `identity:get_user(user_id=dffa20f30b714c7580ce966ceba58add)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:57.145371 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9072f5-9eb9-491d-aa4f-406ff1ee8275 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:57.147852 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 120/243] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 2 18:59:56 2022] GET /identity/v3/users/dffa20f30b714c7580ce966ceba58add => generated 297 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 18:59:57.154451 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcada59c-9f95-4aa3-84ca-c9449cac00a1 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:57.325353 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcada59c-9f95-4aa3-84ca-c9449cac00a1 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:57.374837 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eb4a6f-f5ff-4bec-beba-0b7cbf8c49fd None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:57.375169 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eb4a6f-f5ff-4bec-beba-0b7cbf8c49fd None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:57.375457 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eb4a6f-f5ff-4bec-beba-0b7cbf8c49fd None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:57.377537 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 121/244] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 18:59:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 18:59:57.383667 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab393a-660b-4116-b5a3-55ed4c323d2e None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:57.383960 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab393a-660b-4116-b5a3-55ed4c323d2e None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:57.384063 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82b9fae-8155-4bd6-80b2-9ff18016b2c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:57.384266 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab393a-660b-4116-b5a3-55ed4c323d2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:57.384633 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b9fae-8155-4bd6-80b2-9ff18016b2c8 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:57.384919 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b9fae-8155-4bd6-80b2-9ff18016b2c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:57.385200 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b9fae-8155-4bd6-80b2-9ff18016b2c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:57.403913 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-aaab393a-660b-4116-b5a3-55ed4c323d2e None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 18:59:57.405117 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-aaab393a-660b-4116-b5a3-55ed4c323d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 18:59:57.451602 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:57.469863 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82b9fae-8155-4bd6-80b2-9ff18016b2c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:57.472180 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82b9fae-8155-4bd6-80b2-9ff18016b2c8 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:57.518102 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 122/245] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 18:59:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 02 18:59:57.545608 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaab393a-660b-4116-b5a3-55ed4c323d2e None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:57.593254 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:57.593951 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 124/246] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 2 18:59:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 02 18:59:57.600610 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82b9fae-8155-4bd6-80b2-9ff18016b2c8 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:57.720084 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:57.727257 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaab393a-660b-4116-b5a3-55ed4c323d2e None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:57.739835 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82b9fae-8155-4bd6-80b2-9ff18016b2c8 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:57.788012 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48afab5d-b2d0-49d7-89c5-17b03947571f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:57.788214 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712186e9-139b-4857-8734-21d507e12dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:57.788894 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48afab5d-b2d0-49d7-89c5-17b03947571f None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:57.789004 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712186e9-139b-4857-8734-21d507e12dbe None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:57.789345 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48afab5d-b2d0-49d7-89c5-17b03947571f None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:57.789445 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712186e9-139b-4857-8734-21d507e12dbe None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:57.789721 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712186e9-139b-4857-8734-21d507e12dbe None admin] PATH_INFO: `/v3/roles/member` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:57.789816 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48afab5d-b2d0-49d7-89c5-17b03947571f None admin] PATH_INFO: `/v3/projects/5f19550a1b254991ba8eb40da4841fcf` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:57.791089 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48afab5d-b2d0-49d7-89c5-17b03947571f None admin] RBAC: Authorizing `identity:get_project(project_id=5f19550a1b254991ba8eb40da4841fcf)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:57.794018 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48afab5d-b2d0-49d7-89c5-17b03947571f None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:57.802338 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 125/247] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 2 18:59:57 2022] GET /identity/v3/projects/5f19550a1b254991ba8eb40da4841fcf => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 18:59:57.803710 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48afab5d-b2d0-49d7-89c5-17b03947571f None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:57.810814 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712186e9-139b-4857-8734-21d507e12dbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 18:59:57.822885 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-712186e9-139b-4857-8734-21d507e12dbe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 02 18:59:57.822885 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 18:59:57.822885 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 18:59:57.822885 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 18:59:57.822885 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 18:59:57.822885 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 18:59:57.822885 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 18:59:57.822885 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 18:59:57.822885 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 18:59:57.822885 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 18:59:57.822885 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 18:59:57.822885 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 18:59:57.822885 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 18:59:57.822885 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 18:59:57.822885 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 18:59:57.822885 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 18:59:57.824032 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 18:59:57.824032 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 18:59:57.824032 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 18:59:57.824032 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 18:59:57.824032 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 18:59:57.824032 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 18:59:57.824032 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 18:59:57.824032 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 18:59:57.824032 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 18:59:57.824032 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 18:59:57.824032 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 18:59:57.824032 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 18:59:57.824032 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 18:59:57.824032 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 18:59:57.824032 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 18:59:57.824032 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 18:59:57.825424 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 18:59:57.825424 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 18:59:57.825424 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 18:59:57.825424 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 18:59:57.825424 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 18:59:57.825424 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 18:59:57.825424 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 18:59:57.825424 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 18:59:57.825424 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 18:59:57.825424 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 18:59:57.825424 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 02 18:59:57.825424 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 18:59:57.825424 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 123/248] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 2 18:59:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 18:59:57.831426 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712186e9-139b-4857-8734-21d507e12dbe None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:57.946389 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48afab5d-b2d0-49d7-89c5-17b03947571f None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:57.967253 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:57.991547 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712186e9-139b-4857-8734-21d507e12dbe None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:57.998076 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9cd316-136f-44db-ba86-ec2078fb31a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:58.002847 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9cd316-136f-44db-ba86-ec2078fb31a0 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:58.003383 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9cd316-136f-44db-ba86-ec2078fb31a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:58.003844 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9cd316-136f-44db-ba86-ec2078fb31a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:58.005357 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9cd316-136f-44db-ba86-ec2078fb31a0 None admin] RBAC: Adding query filter params (role.id=de9eea600632474d8b149397589c3746, scope.project.id=5f19550a1b254991ba8eb40da4841fcf, user.id=dffa20f30b714c7580ce966ceba58add) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 18:59:58.005897 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9cd316-136f-44db-ba86-ec2078fb31a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:58.012717 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9cd316-136f-44db-ba86-ec2078fb31a0 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:58.038092 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 126/249] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 2 18:59:57 2022] GET /identity/v3/role_assignments?user.id=dffa20f30b714c7580ce966ceba58add&scope.project.id=5f19550a1b254991ba8eb40da4841fcf&role.id=de9eea600632474d8b149397589c3746 => generated 259 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:58.050786 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c10953-f223-47b9-8be0-6e6346ad35f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:58.052103 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c10953-f223-47b9-8be0-6e6346ad35f7 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:58.052585 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c10953-f223-47b9-8be0-6e6346ad35f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:58.053585 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c10953-f223-47b9-8be0-6e6346ad35f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:58.055060 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c10953-f223-47b9-8be0-6e6346ad35f7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 18:59:58.055625 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c10953-f223-47b9-8be0-6e6346ad35f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:58.058138 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c10953-f223-47b9-8be0-6e6346ad35f7 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:58.079276 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 124/250] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 2 18:59:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 18:59:58.083962 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9cd316-136f-44db-ba86-ec2078fb31a0 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:58.119720 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:58.263880 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9cd316-136f-44db-ba86-ec2078fb31a0 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:58.321148 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b16d785-69a0-401e-8005-2e8d87680281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:58.322066 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b16d785-69a0-401e-8005-2e8d87680281 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:58.322351 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b16d785-69a0-401e-8005-2e8d87680281 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:58.322627 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b16d785-69a0-401e-8005-2e8d87680281 None admin] PATH_INFO: `/v3/users/7aae3b53a99b47378ed66d7870667d89` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:58.324015 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b16d785-69a0-401e-8005-2e8d87680281 None admin] RBAC: Authorizing `identity:get_user(user_id=7aae3b53a99b47378ed66d7870667d89)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:58.325514 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b16d785-69a0-401e-8005-2e8d87680281 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:58.328426 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 127/251] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 2 18:59:58 2022] GET /identity/v3/users/7aae3b53a99b47378ed66d7870667d89 => generated 297 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:58.333001 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c10953-f223-47b9-8be0-6e6346ad35f7 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:58.453726 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c10953-f223-47b9-8be0-6e6346ad35f7 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:58.494066 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e9404c-b661-467f-b08d-8f604a8a1b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:58.495003 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9404c-b661-467f-b08d-8f604a8a1b54 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:58.495294 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9404c-b661-467f-b08d-8f604a8a1b54 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:58.495575 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9404c-b661-467f-b08d-8f604a8a1b54 None admin] PATH_INFO: `/v3/projects/5f19550a1b254991ba8eb40da4841fcf` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:58.496981 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e9404c-b661-467f-b08d-8f604a8a1b54 None admin] RBAC: Authorizing `identity:get_project(project_id=5f19550a1b254991ba8eb40da4841fcf)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:58.498480 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e9404c-b661-467f-b08d-8f604a8a1b54 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:58.500723 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 125/252] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 2 18:59:58 2022] GET /identity/v3/projects/5f19550a1b254991ba8eb40da4841fcf => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 18:59:58.506912 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b16d785-69a0-401e-8005-2e8d87680281 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:58.520076 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99843276-2aae-4978-a0a2-457d160ab9d8 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:58.520584 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99843276-2aae-4978-a0a2-457d160ab9d8 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:58.520584 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99843276-2aae-4978-a0a2-457d160ab9d8 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:58.522654 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 126/253] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 18:59:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 02 18:59:58.528552 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4cfc72-c73a-4377-b3a1-c06e9247d555 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:58.528845 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4cfc72-c73a-4377-b3a1-c06e9247d555 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:58.529122 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4cfc72-c73a-4377-b3a1-c06e9247d555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:58.559398 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-dc4cfc72-c73a-4377-b3a1-c06e9247d555 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 18:59:58.561671 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-dc4cfc72-c73a-4377-b3a1-c06e9247d555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 18:59:58.591123 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:58.660452 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:58.679843 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b16d785-69a0-401e-8005-2e8d87680281 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:58.720598 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afee3c9a-3eea-498c-86e8-985010d65aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:58.721525 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee3c9a-3eea-498c-86e8-985010d65aba None admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:58.721814 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee3c9a-3eea-498c-86e8-985010d65aba None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:58.722099 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee3c9a-3eea-498c-86e8-985010d65aba None admin] PATH_INFO: `/v3/projects/5f19550a1b254991ba8eb40da4841fcf/users/7aae3b53a99b47378ed66d7870667d89/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:58.724350 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afee3c9a-3eea-498c-86e8-985010d65aba None admin] RBAC: Authorizing `identity:create_grant(project_id=5f19550a1b254991ba8eb40da4841fcf, user_id=7aae3b53a99b47378ed66d7870667d89, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:58.726562 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afee3c9a-3eea-498c-86e8-985010d65aba None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:58.745166 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 127/254] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 18:59:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 02 18:59:58.746532 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a646fe-f3f2-4eff-bb7c-17f373987a97 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:58.746927 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a646fe-f3f2-4eff-bb7c-17f373987a97 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:58.747213 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a646fe-f3f2-4eff-bb7c-17f373987a97 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:58.749021 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 128/255] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 18:59:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 18:59:58.755223 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb61d57-9896-45f8-991d-5e49fbd80b58 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:58.755535 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb61d57-9896-45f8-991d-5e49fbd80b58 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:58.755886 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb61d57-9896-45f8-991d-5e49fbd80b58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:58.777926 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-3fb61d57-9896-45f8-991d-5e49fbd80b58 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 18:59:58.777926 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-3fb61d57-9896-45f8-991d-5e49fbd80b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 18:59:58.855646 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 129/256] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 18:59:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 02 18:59:58.856402 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb61d57-9896-45f8-991d-5e49fbd80b58 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:58.865027 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 128/257] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 2 18:59:58 2022] PUT /identity/v3/projects/5f19550a1b254991ba8eb40da4841fcf/users/7aae3b53a99b47378ed66d7870667d89/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 18:59:58.898053 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afee3c9a-3eea-498c-86e8-985010d65aba None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:59.029072 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:59.076829 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb61d57-9896-45f8-991d-5e49fbd80b58 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:59.137806 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afee3c9a-3eea-498c-86e8-985010d65aba None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:59.183510 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e143c600-4160-4112-aa36-ae32ec776a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:59.184744 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e143c600-4160-4112-aa36-ae32ec776a43 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:59.185226 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e143c600-4160-4112-aa36-ae32ec776a43 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:59.185696 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e143c600-4160-4112-aa36-ae32ec776a43 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:59.205201 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83a0a78-2c0c-4164-8df3-cb59e070b932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:59.206047 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a0a78-2c0c-4164-8df3-cb59e070b932 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:59.206338 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a0a78-2c0c-4164-8df3-cb59e070b932 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:59.206619 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a0a78-2c0c-4164-8df3-cb59e070b932 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:59.214033 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e143c600-4160-4112-aa36-ae32ec776a43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 18:59:59.230784 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83a0a78-2c0c-4164-8df3-cb59e070b932 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 18:59:59.231183 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-e143c600-4160-4112-aa36-ae32ec776a43 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 02 18:59:59.231183 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 18:59:59.231183 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 18:59:59.231183 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 18:59:59.231183 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 18:59:59.231183 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 18:59:59.231183 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 18:59:59.231183 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 18:59:59.231183 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 18:59:59.231183 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 18:59:59.231183 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 18:59:59.231183 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 18:59:59.231183 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 18:59:59.231183 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 18:59:59.231183 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 18:59:59.231183 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 18:59:59.232011 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 18:59:59.232011 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 18:59:59.232011 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 18:59:59.232011 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 18:59:59.232011 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 18:59:59.232011 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 18:59:59.232011 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 18:59:59.232011 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 18:59:59.232011 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 18:59:59.232011 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 18:59:59.232011 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 18:59:59.232011 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 18:59:59.232011 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 18:59:59.232011 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 18:59:59.232011 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 18:59:59.232011 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 18:59:59.232790 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 18:59:59.232790 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 18:59:59.232790 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 18:59:59.232790 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 18:59:59.232790 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 18:59:59.232790 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 18:59:59.232790 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 18:59:59.232790 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 18:59:59.232790 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 18:59:59.232790 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 18:59:59.232790 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 02 18:59:59.232790 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 18:59:59.235356 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 130/258] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 2 18:59:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 18:59:59.241755 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-a83a0a78-2c0c-4164-8df3-cb59e070b932 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 02 18:59:59.241755 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 18:59:59.241755 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 18:59:59.241755 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 18:59:59.241755 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 18:59:59.241755 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 18:59:59.241755 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 18:59:59.241755 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 18:59:59.241755 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 18:59:59.241755 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 18:59:59.241755 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 18:59:59.241755 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 18:59:59.241755 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 18:59:59.241755 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 18:59:59.241755 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 18:59:59.241755 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 18:59:59.242668 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 18:59:59.242668 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 18:59:59.242668 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 18:59:59.242668 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 18:59:59.242668 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 18:59:59.242668 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 18:59:59.242668 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 18:59:59.242668 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 18:59:59.242668 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 18:59:59.242668 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 18:59:59.242668 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 18:59:59.242668 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 18:59:59.242668 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 18:59:59.242668 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 18:59:59.242668 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 18:59:59.242668 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 18:59:59.243881 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 18:59:59.243881 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 18:59:59.243881 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 18:59:59.243881 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 18:59:59.243881 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 18:59:59.243881 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 18:59:59.243881 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 18:59:59.243881 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 18:59:59.243881 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 18:59:59.243881 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 18:59:59.243881 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 02 18:59:59.243881 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 18:59:59.243881 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 129/259] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 2 18:59:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 02 18:59:59.245107 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e143c600-4160-4112-aa36-ae32ec776a43 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:59.251007 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83a0a78-2c0c-4164-8df3-cb59e070b932 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:59.267650 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:59.451664 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83a0a78-2c0c-4164-8df3-cb59e070b932 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:59.486140 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e143c600-4160-4112-aa36-ae32ec776a43 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:59.518477 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03289800-266c-416b-8bd4-8e52698dac26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:59.519534 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03289800-266c-416b-8bd4-8e52698dac26 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:59.520019 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03289800-266c-416b-8bd4-8e52698dac26 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:59.520493 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03289800-266c-416b-8bd4-8e52698dac26 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:59.521460 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03289800-266c-416b-8bd4-8e52698dac26 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 18:59:59.522019 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03289800-266c-416b-8bd4-8e52698dac26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:59.523645 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03289800-266c-416b-8bd4-8e52698dac26 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:59.537029 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 130/260] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 2 18:59:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 18:59:59.544339 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03289800-266c-416b-8bd4-8e52698dac26 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:59.552020 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294fbb24-d689-4b30-bda0-d5f8a5710bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:59.552709 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294fbb24-d689-4b30-bda0-d5f8a5710bd4 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:59.553534 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294fbb24-d689-4b30-bda0-d5f8a5710bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:59.553817 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294fbb24-d689-4b30-bda0-d5f8a5710bd4 None admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:59.554584 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294fbb24-d689-4b30-bda0-d5f8a5710bd4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 18:59:59.558789 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294fbb24-d689-4b30-bda0-d5f8a5710bd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:59.558789 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294fbb24-d689-4b30-bda0-d5f8a5710bd4 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:59.559748 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:59.573950 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 131/261] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 2 18:59:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 18:59:59.578385 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294fbb24-d689-4b30-bda0-d5f8a5710bd4 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:59.652734 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 18:59:59.718682 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03289800-266c-416b-8bd4-8e52698dac26 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:59.778056 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f31fc2f-3fe4-4eef-9338-08a205605f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:59.779318 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f31fc2f-3fe4-4eef-9338-08a205605f57 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:59.779812 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f31fc2f-3fe4-4eef-9338-08a205605f57 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:59.780265 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f31fc2f-3fe4-4eef-9338-08a205605f57 None admin] PATH_INFO: `/v3/users/f830a676315f4e5c85ca846dfa43459d` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:59.786990 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f31fc2f-3fe4-4eef-9338-08a205605f57 None admin] RBAC: Authorizing `identity:get_user(user_id=f830a676315f4e5c85ca846dfa43459d)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:59.787219 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294fbb24-d689-4b30-bda0-d5f8a5710bd4 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:59.789689 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f31fc2f-3fe4-4eef-9338-08a205605f57 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:59.792091 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 131/262] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 2 18:59:59 2022] GET /identity/v3/users/f830a676315f4e5c85ca846dfa43459d => generated 283 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 18:59:59.797338 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f31fc2f-3fe4-4eef-9338-08a205605f57 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:59.860392 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a5a56e-07df-4644-926c-f761b1ad381f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 18:59:59.861563 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a5a56e-07df-4644-926c-f761b1ad381f None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 18:59:59.861997 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a5a56e-07df-4644-926c-f761b1ad381f None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 18:59:59.862417 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a5a56e-07df-4644-926c-f761b1ad381f None admin] PATH_INFO: `/v3/users/f830a676315f4e5c85ca846dfa43459d` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 18:59:59.867496 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a5a56e-07df-4644-926c-f761b1ad381f None admin] RBAC: Authorizing `identity:get_user(user_id=f830a676315f4e5c85ca846dfa43459d)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 18:59:59.869142 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a5a56e-07df-4644-926c-f761b1ad381f None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 18:59:59.871045 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 132/263] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 2 18:59:59 2022] GET /identity/v3/users/f830a676315f4e5c85ca846dfa43459d => generated 283 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 18:59:59.879632 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a5a56e-07df-4644-926c-f761b1ad381f None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 18:59:59.978292 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f31fc2f-3fe4-4eef-9338-08a205605f57 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 18:59:59.998860 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:00.027466 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e29b199-7c4b-4951-bd30-1fabef936040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:00.028744 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e29b199-7c4b-4951-bd30-1fabef936040 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:00.029252 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e29b199-7c4b-4951-bd30-1fabef936040 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:00.029722 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e29b199-7c4b-4951-bd30-1fabef936040 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:00.033011 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:00.051005 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a5a56e-07df-4644-926c-f761b1ad381f None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:00.135640 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f307f6b6-9abc-4845-b0c4-e3973d05e800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:00.136342 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f307f6b6-9abc-4845-b0c4-e3973d05e800 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:00.136630 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f307f6b6-9abc-4845-b0c4-e3973d05e800 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:00.136912 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f307f6b6-9abc-4845-b0c4-e3973d05e800 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:00.142930 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e29b199-7c4b-4951-bd30-1fabef936040 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:00.144447 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e29b199-7c4b-4951-bd30-1fabef936040 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:00.261086 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f307f6b6-9abc-4845-b0c4-e3973d05e800 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:00.262518 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f307f6b6-9abc-4845-b0c4-e3973d05e800 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:00.362320 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:00.373927 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 132/264] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 2 18:59:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 577 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 02 19:00:00.381983 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e29b199-7c4b-4951-bd30-1fabef936040 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:00.430455 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 133/265] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 2 18:59:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 551 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 02 19:00:00.436064 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f307f6b6-9abc-4845-b0c4-e3973d05e800 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:00.455165 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:00.612345 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f307f6b6-9abc-4845-b0c4-e3973d05e800 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:00.629329 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e29b199-7c4b-4951-bd30-1fabef936040 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:00.675121 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9aa7c5-d290-430a-9a9a-0b11674f7c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:00.676461 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9aa7c5-d290-430a-9a9a-0b11674f7c75 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:00.676948 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9aa7c5-d290-430a-9a9a-0b11674f7c75 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:00.677446 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9aa7c5-d290-430a-9a9a-0b11674f7c75 None admin] PATH_INFO: `/v3/projects/5f19550a1b254991ba8eb40da4841fcf` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:00.679630 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9aa7c5-d290-430a-9a9a-0b11674f7c75 None admin] RBAC: Authorizing `identity:get_project(project_id=5f19550a1b254991ba8eb40da4841fcf)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:00.681296 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9aa7c5-d290-430a-9a9a-0b11674f7c75 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:00.692326 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 134/266] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 2 19:00:00 2022] GET /identity/v3/projects/5f19550a1b254991ba8eb40da4841fcf => generated 302 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:00:00.698439 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9aa7c5-d290-430a-9a9a-0b11674f7c75 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:00.723519 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad174327-d8bc-4836-8114-80c917bf49c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:00.724422 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad174327-d8bc-4836-8114-80c917bf49c8 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:00.724708 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad174327-d8bc-4836-8114-80c917bf49c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:00.725522 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad174327-d8bc-4836-8114-80c917bf49c8 None admin] PATH_INFO: `/v3/projects/5f19550a1b254991ba8eb40da4841fcf` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:00.726845 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad174327-d8bc-4836-8114-80c917bf49c8 None admin] RBAC: Authorizing `identity:get_project(project_id=5f19550a1b254991ba8eb40da4841fcf)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:00.728309 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad174327-d8bc-4836-8114-80c917bf49c8 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:00.731488 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 133/267] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 2 19:00:00 2022] GET /identity/v3/projects/5f19550a1b254991ba8eb40da4841fcf => generated 302 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:00:00.736761 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad174327-d8bc-4836-8114-80c917bf49c8 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:00.760386 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:00.885542 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad174327-d8bc-4836-8114-80c917bf49c8 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:00.888200 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:00.890684 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9aa7c5-d290-430a-9a9a-0b11674f7c75 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:00.955403 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd37310c-ad99-466e-9896-7d8bccf346c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:00.956073 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd37310c-ad99-466e-9896-7d8bccf346c6 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:00.956355 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd37310c-ad99-466e-9896-7d8bccf346c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:00.956629 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd37310c-ad99-466e-9896-7d8bccf346c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:00.957471 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd37310c-ad99-466e-9896-7d8bccf346c6 None admin] RBAC: Adding query filter params (role.id=f13fdd0847924014abf1e95594195e78, scope.project.id=5f19550a1b254991ba8eb40da4841fcf, user.id=f830a676315f4e5c85ca846dfa43459d) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:00.957784 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd37310c-ad99-466e-9896-7d8bccf346c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:00.961342 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd37310c-ad99-466e-9896-7d8bccf346c6 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:00.969239 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d06154-0502-4dbf-ad5d-df4dc1be5f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:00.969911 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d06154-0502-4dbf-ad5d-df4dc1be5f05 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:00.970195 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d06154-0502-4dbf-ad5d-df4dc1be5f05 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:00.970471 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d06154-0502-4dbf-ad5d-df4dc1be5f05 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:00.971350 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d06154-0502-4dbf-ad5d-df4dc1be5f05 None admin] RBAC: Adding query filter params (role.id=2e0c054b646e4ed2a0e7e5962f09a50e, scope.project.id=5f19550a1b254991ba8eb40da4841fcf, user.id=f830a676315f4e5c85ca846dfa43459d) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:00.971661 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d06154-0502-4dbf-ad5d-df4dc1be5f05 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:00.973116 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d06154-0502-4dbf-ad5d-df4dc1be5f05 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:00.979252 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 134/268] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 2 19:00:00 2022] GET /identity/v3/role_assignments?user.id=f830a676315f4e5c85ca846dfa43459d&scope.project.id=5f19550a1b254991ba8eb40da4841fcf&role.id=f13fdd0847924014abf1e95594195e78 => generated 608 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:00:00.984714 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 135/269] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 2 19:00:00 2022] GET /identity/v3/role_assignments?user.id=f830a676315f4e5c85ca846dfa43459d&scope.project.id=5f19550a1b254991ba8eb40da4841fcf&role.id=2e0c054b646e4ed2a0e7e5962f09a50e => generated 608 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:00:01.220198 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57acfac3-80e2-4a16-9b82-81771a5701e6 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:01.220587 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57acfac3-80e2-4a16-9b82-81771a5701e6 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:01.220766 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57acfac3-80e2-4a16-9b82-81771a5701e6 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:01.222561 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 135/270] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:00:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:00:01.229134 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccda812-2b1d-48ec-849a-5afc9eae0a49 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:01.229436 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccda812-2b1d-48ec-849a-5afc9eae0a49 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:01.229702 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccda812-2b1d-48ec-849a-5afc9eae0a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:01.257157 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-3ccda812-2b1d-48ec-849a-5afc9eae0a49 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:00:01.258793 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-3ccda812-2b1d-48ec-849a-5afc9eae0a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:00:01.259948 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686d2e7-93bb-446c-9fa0-204b32a55596 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:01.260237 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686d2e7-93bb-446c-9fa0-204b32a55596 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:01.260515 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686d2e7-93bb-446c-9fa0-204b32a55596 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:01.262264 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 136/271] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:00:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:00:01.267611 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3b44ac-50eb-48e4-85ab-1d338979bb74 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:01.267898 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3b44ac-50eb-48e4-85ab-1d338979bb74 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:01.268178 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3b44ac-50eb-48e4-85ab-1d338979bb74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:01.299395 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-3a3b44ac-50eb-48e4-85ab-1d338979bb74 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:00:01.300689 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-3a3b44ac-50eb-48e4-85ab-1d338979bb74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:00:01.339913 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 136/272] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:00:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 19:00:01.375560 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccda812-2b1d-48ec-849a-5afc9eae0a49 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:01.392728 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 137/273] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:00:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 02 19:00:01.419766 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3b44ac-50eb-48e4-85ab-1d338979bb74 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:01.426303 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:01.534205 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:01.536465 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccda812-2b1d-48ec-849a-5afc9eae0a49 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:01.540968 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3b44ac-50eb-48e4-85ab-1d338979bb74 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:01.581239 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b82aca-30ca-4ce0-827b-42a62bfc64e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:01.582074 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b82aca-30ca-4ce0-827b-42a62bfc64e7 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:01.582421 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b82aca-30ca-4ce0-827b-42a62bfc64e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:01.582692 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b82aca-30ca-4ce0-827b-42a62bfc64e7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:01.593405 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848ec9d0-79bc-44d4-9682-3dc91edc7849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:01.594205 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ec9d0-79bc-44d4-9682-3dc91edc7849 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:01.594489 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ec9d0-79bc-44d4-9682-3dc91edc7849 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:01.594727 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b82aca-30ca-4ce0-827b-42a62bfc64e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:00:01.594915 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ec9d0-79bc-44d4-9682-3dc91edc7849 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:01.595981 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848ec9d0-79bc-44d4-9682-3dc91edc7849 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:01.597456 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848ec9d0-79bc-44d4-9682-3dc91edc7849 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:01.600494 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 138/274] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:00:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:01.601325 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b20d3c-fe20-49a3-a371-011a1fe8f84f None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:01.601608 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b20d3c-fe20-49a3-a371-011a1fe8f84f None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:01.601885 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b20d3c-fe20-49a3-a371-011a1fe8f84f None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:01.605842 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-72b82aca-30ca-4ce0-827b-42a62bfc64e7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 02 19:00:01.605842 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:01.605842 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:01.605842 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:01.605842 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:01.605842 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:01.605842 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:01.605842 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:01.605842 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:01.605842 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:01.605842 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:01.605842 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:01.605842 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:00:01.605842 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:00:01.605842 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:00:01.605842 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:00:01.606731 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:00:01.606731 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:00:01.606731 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:00:01.606731 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:00:01.606731 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:01.606731 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:01.606731 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:01.606731 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:01.606731 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:01.606731 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:01.606731 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:01.606731 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:01.606731 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:01.606731 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:01.606731 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:01.606731 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:01.607872 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:01.607872 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:01.607872 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:01.607872 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:01.607872 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:00:01.607872 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:00:01.607872 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:00:01.607872 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:00:01.607872 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:00:01.607872 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:00:01.607872 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 02 19:00:01.607872 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:00:01.608470 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 139/275] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:00:01 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:00:01.609063 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 137/276] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 2 19:00:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:01.609262 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b20d3c-fe20-49a3-a371-011a1fe8f84f None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:01.615018 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1736398a-a96c-40f3-9657-5496c153887c None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:01.615569 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1736398a-a96c-40f3-9657-5496c153887c None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:01.615642 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1736398a-a96c-40f3-9657-5496c153887c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:01.633226 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-1736398a-a96c-40f3-9657-5496c153887c None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:00:01.634432 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-1736398a-a96c-40f3-9657-5496c153887c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:00:01.711300 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 138/277] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:00:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 19:00:01.711300 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1736398a-a96c-40f3-9657-5496c153887c None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:01.725817 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b20d3c-fe20-49a3-a371-011a1fe8f84f None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:01.737609 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:01.764513 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27284df2-7712-4948-b293-6d7344c7db34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:01.765249 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27284df2-7712-4948-b293-6d7344c7db34 None admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:01.765461 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27284df2-7712-4948-b293-6d7344c7db34 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:01.765837 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27284df2-7712-4948-b293-6d7344c7db34 None admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:01.766678 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27284df2-7712-4948-b293-6d7344c7db34 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:01.786965 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27284df2-7712-4948-b293-6d7344c7db34 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:01.824149 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1736398a-a96c-40f3-9657-5496c153887c None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:01.859687 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242533a9-3e90-43a0-a9b4-17a2e80d83b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:01.860374 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242533a9-3e90-43a0-a9b4-17a2e80d83b2 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:01.860662 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242533a9-3e90-43a0-a9b4-17a2e80d83b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:01.860934 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242533a9-3e90-43a0-a9b4-17a2e80d83b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:01.861708 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242533a9-3e90-43a0-a9b4-17a2e80d83b2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:01.862018 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242533a9-3e90-43a0-a9b4-17a2e80d83b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:01.863518 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242533a9-3e90-43a0-a9b4-17a2e80d83b2 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:01.873379 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 139/278] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 2 19:00:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:01.874307 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242533a9-3e90-43a0-a9b4-17a2e80d83b2 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:01.893740 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 140/279] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 2 19:00:01 2022] POST /identity/v3/users => generated 293 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:00:01.903932 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27284df2-7712-4948-b293-6d7344c7db34 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:01.945752 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:01.987316 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:02.010811 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242533a9-3e90-43a0-a9b4-17a2e80d83b2 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:02.063236 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27284df2-7712-4948-b293-6d7344c7db34 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:02.064209 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879d721c-a692-409f-a923-32ff70f88669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:02.065223 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d721c-a692-409f-a923-32ff70f88669 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:02.065684 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d721c-a692-409f-a923-32ff70f88669 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:02.066130 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d721c-a692-409f-a923-32ff70f88669 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:02.079071 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879d721c-a692-409f-a923-32ff70f88669 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:00:02.088017 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-879d721c-a692-409f-a923-32ff70f88669 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 02 19:00:02.088017 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:02.088017 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:02.088017 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:02.088017 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:02.088017 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:02.088017 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:02.088017 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:02.088017 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:02.088017 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:02.088017 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:02.088017 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:02.088017 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:00:02.088017 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:00:02.088017 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:00:02.088017 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:00:02.088975 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:00:02.088975 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:00:02.088975 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:00:02.088975 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:00:02.088975 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:02.088975 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:02.088975 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:02.088975 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:02.088975 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:02.088975 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:02.088975 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:02.088975 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:02.088975 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:02.088975 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:02.088975 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:02.088975 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:02.089759 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:02.089759 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:02.089759 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:02.089759 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:02.089759 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:00:02.089759 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:00:02.089759 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:00:02.089759 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:00:02.089759 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:00:02.089759 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:00:02.089759 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 02 19:00:02.089759 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:00:02.092462 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 140/280] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 2 19:00:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:02.101944 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879d721c-a692-409f-a923-32ff70f88669 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:02.110318 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef4da08-2e04-435e-aec5-8c6044fec7c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:02.112049 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef4da08-2e04-435e-aec5-8c6044fec7c9 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:02.112049 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef4da08-2e04-435e-aec5-8c6044fec7c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:02.112049 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef4da08-2e04-435e-aec5-8c6044fec7c9 None admin] PATH_INFO: `/v3/users/5d028041b39a4c91ad4022d4b766ada3` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:02.134505 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef4da08-2e04-435e-aec5-8c6044fec7c9 None admin] RBAC: Authorizing `identity:get_user(user_id=5d028041b39a4c91ad4022d4b766ada3)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:02.136593 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef4da08-2e04-435e-aec5-8c6044fec7c9 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:02.138803 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 141/281] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 2 19:00:01 2022] GET /identity/v3/users/5d028041b39a4c91ad4022d4b766ada3 => generated 293 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:02.144808 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef4da08-2e04-435e-aec5-8c6044fec7c9 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:02.237999 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:02.252604 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879d721c-a692-409f-a923-32ff70f88669 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:02.268926 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef4da08-2e04-435e-aec5-8c6044fec7c9 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:02.297436 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48814df5-7f30-453f-abb2-3282ed2e4d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:02.298123 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48814df5-7f30-453f-abb2-3282ed2e4d89 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:02.298413 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48814df5-7f30-453f-abb2-3282ed2e4d89 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:02.298971 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48814df5-7f30-453f-abb2-3282ed2e4d89 None admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:02.299526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48814df5-7f30-453f-abb2-3282ed2e4d89 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:02.299836 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48814df5-7f30-453f-abb2-3282ed2e4d89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:02.301215 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48814df5-7f30-453f-abb2-3282ed2e4d89 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:02.313150 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 141/282] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 2 19:00:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:02.314601 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f14cac5-7d0a-4d6e-b4ec-a4fee02a6f44 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:02.314875 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f14cac5-7d0a-4d6e-b4ec-a4fee02a6f44 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:02.315105 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153b7299-7cc4-4f18-9912-0097c7cfe8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:02.315219 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f14cac5-7d0a-4d6e-b4ec-a4fee02a6f44 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:02.315893 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153b7299-7cc4-4f18-9912-0097c7cfe8e6 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:02.316441 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153b7299-7cc4-4f18-9912-0097c7cfe8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:02.316441 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153b7299-7cc4-4f18-9912-0097c7cfe8e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:02.316572 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 142/283] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:00:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:00:02.317378 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153b7299-7cc4-4f18-9912-0097c7cfe8e6 None admin] RBAC: Adding query filter params (role.id=77d5301ed78146468d4115b512cc70b7, user.id=5d028041b39a4c91ad4022d4b766ada3, scope.system=all) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:02.317781 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153b7299-7cc4-4f18-9912-0097c7cfe8e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:02.319864 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153b7299-7cc4-4f18-9912-0097c7cfe8e6 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:02.320363 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f14cac5-7d0a-4d6e-b4ec-a4fee02a6f44 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:02.323503 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:02.327579 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 142/284] 10.222.0.33 () {56 vars in 1308 bytes} [Thu Jun 2 19:00:02 2022] GET /identity/v3/role_assignments?user.id=5d028041b39a4c91ad4022d4b766ada3&scope.system=all&role.id=77d5301ed78146468d4115b512cc70b7 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:00:02.328935 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea49d0-62aa-48c4-b72e-f352e20169e0 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:02.329189 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea49d0-62aa-48c4-b72e-f352e20169e0 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:02.329467 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea49d0-62aa-48c4-b72e-f352e20169e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:02.350482 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-c4ea49d0-62aa-48c4-b72e-f352e20169e0 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:00:02.352647 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-c4ea49d0-62aa-48c4-b72e-f352e20169e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:00:02.453321 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 143/285] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:00:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 19:00:02.483693 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ea49d0-62aa-48c4-b72e-f352e20169e0 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:02.497329 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f14cac5-7d0a-4d6e-b4ec-a4fee02a6f44 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:02.569555 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f98d16-1ca7-41bd-b457-276beeef40ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:02.570388 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f98d16-1ca7-41bd-b457-276beeef40ba None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:02.570679 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f98d16-1ca7-41bd-b457-276beeef40ba None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:02.570992 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f98d16-1ca7-41bd-b457-276beeef40ba None admin] PATH_INFO: `/v3/users/dffa20f30b714c7580ce966ceba58add` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:02.573074 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f98d16-1ca7-41bd-b457-276beeef40ba None admin] RBAC: Authorizing `identity:get_user(user_id=dffa20f30b714c7580ce966ceba58add)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:02.574903 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f98d16-1ca7-41bd-b457-276beeef40ba None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:02.577573 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 143/286] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 2 19:00:02 2022] GET /identity/v3/users/dffa20f30b714c7580ce966ceba58add => generated 297 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:02.583295 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f98d16-1ca7-41bd-b457-276beeef40ba None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:02.621126 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:02.674645 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:02.686300 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ea49d0-62aa-48c4-b72e-f352e20169e0 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:02.708204 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f98d16-1ca7-41bd-b457-276beeef40ba None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:02.748445 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb408e5-d724-461e-b550-1596db59c7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:02.749442 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb408e5-d724-461e-b550-1596db59c7bd None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:02.749737 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb408e5-d724-461e-b550-1596db59c7bd None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:02.750015 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb408e5-d724-461e-b550-1596db59c7bd None admin] PATH_INFO: `/v3/projects/5f19550a1b254991ba8eb40da4841fcf` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:02.751250 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb408e5-d724-461e-b550-1596db59c7bd None admin] RBAC: Authorizing `identity:get_project(project_id=5f19550a1b254991ba8eb40da4841fcf)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:02.753078 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb408e5-d724-461e-b550-1596db59c7bd None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:02.755140 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 144/287] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 2 19:00:02 2022] GET /identity/v3/projects/5f19550a1b254991ba8eb40da4841fcf => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:02.755768 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f3aabf-ad10-4a1c-b40c-a282e5cf966e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:02.757615 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f3aabf-ad10-4a1c-b40c-a282e5cf966e None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:02.758075 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f3aabf-ad10-4a1c-b40c-a282e5cf966e None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:02.758507 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f3aabf-ad10-4a1c-b40c-a282e5cf966e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:02.761295 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb408e5-d724-461e-b550-1596db59c7bd None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:02.778515 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f3aabf-ad10-4a1c-b40c-a282e5cf966e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:00:02.789343 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-33f3aabf-ad10-4a1c-b40c-a282e5cf966e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 02 19:00:02.789343 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:02.789343 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:02.789343 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:02.789343 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:02.789343 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:02.789343 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:02.789343 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:02.789343 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:02.789343 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:02.789343 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:02.789343 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:02.789343 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:00:02.789343 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:00:02.789343 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:00:02.789343 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:00:02.790256 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:00:02.790256 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:00:02.790256 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:00:02.790256 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:00:02.790256 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:02.790256 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:02.790256 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:02.790256 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:02.790256 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:02.790256 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:02.790256 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:02.790256 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:02.790256 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:02.790256 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:02.790256 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:02.790256 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:02.791081 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:02.791081 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:02.791081 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:02.791081 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:02.791081 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:00:02.791081 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:00:02.791081 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:00:02.791081 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:00:02.791081 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:00:02.791081 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:00:02.791081 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 02 19:00:02.791081 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:00:02.795016 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 144/288] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 2 19:00:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 02 19:00:02.796716 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f3aabf-ad10-4a1c-b40c-a282e5cf966e None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:02.876039 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb408e5-d724-461e-b550-1596db59c7bd None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:02.903114 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f3aabf-ad10-4a1c-b40c-a282e5cf966e None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:02.915551 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ecb778-6b63-4b36-9098-e715cdcb649d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:02.916461 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ecb778-6b63-4b36-9098-e715cdcb649d None admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:02.916850 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ecb778-6b63-4b36-9098-e715cdcb649d None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:02.917135 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ecb778-6b63-4b36-9098-e715cdcb649d None admin] PATH_INFO: `/v3/projects/5f19550a1b254991ba8eb40da4841fcf/users/dffa20f30b714c7580ce966ceba58add/roles/de9eea600632474d8b149397589c3746` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:02.919320 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ecb778-6b63-4b36-9098-e715cdcb649d None admin] RBAC: Authorizing `identity:create_grant(project_id=5f19550a1b254991ba8eb40da4841fcf, user_id=dffa20f30b714c7580ce966ceba58add, role_id=de9eea600632474d8b149397589c3746)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:02.921007 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ecb778-6b63-4b36-9098-e715cdcb649d None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:02.935553 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:02.939171 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38544b84-3d43-4e39-a148-536448b26eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:02.939876 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38544b84-3d43-4e39-a148-536448b26eeb None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:02.940164 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38544b84-3d43-4e39-a148-536448b26eeb None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:02.940447 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38544b84-3d43-4e39-a148-536448b26eeb None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:02.941231 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38544b84-3d43-4e39-a148-536448b26eeb None admin] RBAC: Adding query filter params (name=member) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:02.941542 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38544b84-3d43-4e39-a148-536448b26eeb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:02.943030 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38544b84-3d43-4e39-a148-536448b26eeb None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:02.957032 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 145/289] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 2 19:00:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:02.958342 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38544b84-3d43-4e39-a148-536448b26eeb None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:02.980787 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:02.993477 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 145/290] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 2 19:00:02 2022] PUT /identity/v3/projects/5f19550a1b254991ba8eb40da4841fcf/users/dffa20f30b714c7580ce966ceba58add/roles/de9eea600632474d8b149397589c3746 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 02 19:00:03.077186 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38544b84-3d43-4e39-a148-536448b26eeb None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:03.130733 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439b929c-6d55-4509-b1dd-2662ef1092be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:03.131637 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439b929c-6d55-4509-b1dd-2662ef1092be None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:03.132026 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439b929c-6d55-4509-b1dd-2662ef1092be None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:03.132315 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439b929c-6d55-4509-b1dd-2662ef1092be None admin] PATH_INFO: `/v3/users/7aae3b53a99b47378ed66d7870667d89` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:03.133740 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439b929c-6d55-4509-b1dd-2662ef1092be None admin] RBAC: Authorizing `identity:get_user(user_id=7aae3b53a99b47378ed66d7870667d89)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:03.135279 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439b929c-6d55-4509-b1dd-2662ef1092be None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:03.137095 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 146/291] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 2 19:00:02 2022] GET /identity/v3/users/7aae3b53a99b47378ed66d7870667d89 => generated 297 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:00:03.142937 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ecb778-6b63-4b36-9098-e715cdcb649d None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:03.302175 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ecb778-6b63-4b36-9098-e715cdcb649d None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:03.342044 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3639f20f-3fdc-4da0-83ec-71b56fb18694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:03.342705 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639f20f-3fdc-4da0-83ec-71b56fb18694 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:03.343036 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639f20f-3fdc-4da0-83ec-71b56fb18694 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:03.343311 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639f20f-3fdc-4da0-83ec-71b56fb18694 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:03.413913 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:03.418137 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3639f20f-3fdc-4da0-83ec-71b56fb18694 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:03.419741 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3639f20f-3fdc-4da0-83ec-71b56fb18694 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:03.576367 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 146/292] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 2 19:00:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 02 19:00:03.582466 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439b929c-6d55-4509-b1dd-2662ef1092be None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:03.708703 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:03.766810 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439b929c-6d55-4509-b1dd-2662ef1092be None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:03.805214 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e7afe7-03e9-4234-8170-ee669ae56481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:03.806145 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e7afe7-03e9-4234-8170-ee669ae56481 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:03.806430 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e7afe7-03e9-4234-8170-ee669ae56481 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:03.806701 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e7afe7-03e9-4234-8170-ee669ae56481 None admin] PATH_INFO: `/v3/projects/5f19550a1b254991ba8eb40da4841fcf` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:03.807941 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e7afe7-03e9-4234-8170-ee669ae56481 None admin] RBAC: Authorizing `identity:get_project(project_id=5f19550a1b254991ba8eb40da4841fcf)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:03.809383 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e7afe7-03e9-4234-8170-ee669ae56481 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:03.811439 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 147/293] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 2 19:00:03 2022] GET /identity/v3/projects/5f19550a1b254991ba8eb40da4841fcf => generated 302 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:03.817160 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3639f20f-3fdc-4da0-83ec-71b56fb18694 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:03.956696 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:03.967159 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3639f20f-3fdc-4da0-83ec-71b56fb18694 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:04.013332 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dd2c01-2463-4df1-a8d1-39bb14e8b12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:04.014000 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dd2c01-2463-4df1-a8d1-39bb14e8b12f None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:04.014281 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dd2c01-2463-4df1-a8d1-39bb14e8b12f None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:04.014550 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dd2c01-2463-4df1-a8d1-39bb14e8b12f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:04.015430 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dd2c01-2463-4df1-a8d1-39bb14e8b12f None admin] RBAC: Adding query filter params (role.id=77d5301ed78146468d4115b512cc70b7, scope.project.id=5f19550a1b254991ba8eb40da4841fcf, user.id=7aae3b53a99b47378ed66d7870667d89) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:04.015740 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dd2c01-2463-4df1-a8d1-39bb14e8b12f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:04.017187 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dd2c01-2463-4df1-a8d1-39bb14e8b12f None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:04.027865 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 147/294] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 2 19:00:03 2022] GET /identity/v3/role_assignments?user.id=7aae3b53a99b47378ed66d7870667d89&scope.project.id=5f19550a1b254991ba8eb40da4841fcf&role.id=77d5301ed78146468d4115b512cc70b7 => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:04.763663 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7116ad33-b086-4f54-bab9-1873f9350aad None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:04.764235 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7116ad33-b086-4f54-bab9-1873f9350aad None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:04.764235 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7116ad33-b086-4f54-bab9-1873f9350aad None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:04.765897 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 148/295] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:00:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:00:04.772709 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a18570-ae18-43a2-9573-39c3c459da37 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:04.772999 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a18570-ae18-43a2-9573-39c3c459da37 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:04.773345 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a18570-ae18-43a2-9573-39c3c459da37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:04.795554 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-60a18570-ae18-43a2-9573-39c3c459da37 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:00:04.796783 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-60a18570-ae18-43a2-9573-39c3c459da37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:00:04.840148 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d000e16-199b-478b-a6bb-8468b263ab07 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:04.840793 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d000e16-199b-478b-a6bb-8468b263ab07 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:04.841302 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d000e16-199b-478b-a6bb-8468b263ab07 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:04.842940 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 149/296] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:00:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 02 19:00:04.851633 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e2633d-fe82-4a97-96f7-a8eec35a5e3e None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:04.852282 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e2633d-fe82-4a97-96f7-a8eec35a5e3e None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:04.852759 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e2633d-fe82-4a97-96f7-a8eec35a5e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:04.873748 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-12e2633d-fe82-4a97-96f7-a8eec35a5e3e None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:00:04.876285 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-12e2633d-fe82-4a97-96f7-a8eec35a5e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:00:04.893496 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 148/297] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:00:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 19:00:04.916776 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a18570-ae18-43a2-9573-39c3c459da37 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:04.962974 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 150/298] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:00:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 02 19:00:04.981763 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e2633d-fe82-4a97-96f7-a8eec35a5e3e None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:05.012622 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:05.021162 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a18570-ae18-43a2-9573-39c3c459da37 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:05.047757 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:05.057638 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227b9eae-ee34-434b-9d5c-7df979b8ca37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:05.058652 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b9eae-ee34-434b-9d5c-7df979b8ca37 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:05.059123 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b9eae-ee34-434b-9d5c-7df979b8ca37 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:05.059613 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b9eae-ee34-434b-9d5c-7df979b8ca37 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:05.060975 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227b9eae-ee34-434b-9d5c-7df979b8ca37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:05.062614 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227b9eae-ee34-434b-9d5c-7df979b8ca37 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:05.064739 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 149/299] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:00:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:00:05.072626 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227b9eae-ee34-434b-9d5c-7df979b8ca37 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:05.099329 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e2633d-fe82-4a97-96f7-a8eec35a5e3e None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:05.158054 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f18f7ca-3d58-4a03-8ee5-832e98c730f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:05.159214 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18f7ca-3d58-4a03-8ee5-832e98c730f8 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:05.159657 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18f7ca-3d58-4a03-8ee5-832e98c730f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:05.160123 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18f7ca-3d58-4a03-8ee5-832e98c730f8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:05.176235 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f18f7ca-3d58-4a03-8ee5-832e98c730f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:00:05.185315 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-6f18f7ca-3d58-4a03-8ee5-832e98c730f8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 02 19:00:05.185315 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:05.185315 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:05.185315 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:05.185315 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:05.185315 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:05.185315 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:05.185315 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:05.185315 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:05.185315 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:05.185315 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:05.185315 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:05.185315 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:00:05.185315 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:00:05.185315 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:00:05.185315 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:00:05.186278 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:00:05.186278 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:00:05.186278 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:00:05.186278 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:00:05.186278 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:05.186278 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:05.186278 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:05.186278 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:05.186278 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:05.186278 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:05.186278 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:05.186278 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:05.186278 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:05.186278 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:05.186278 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:05.186278 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:05.187106 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:05.187106 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:05.187106 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:05.187106 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:05.187106 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:00:05.187106 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:00:05.187106 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:00:05.187106 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:00:05.187106 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:00:05.187106 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:00:05.187106 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 02 19:00:05.187106 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:00:05.187106 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 151/300] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 2 19:00:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 02 19:00:05.192845 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f18f7ca-3d58-4a03-8ee5-832e98c730f8 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:05.222782 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227b9eae-ee34-434b-9d5c-7df979b8ca37 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:05.263975 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6597b9-4d5a-4a2b-b4f2-57a86e9f9f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:05.264647 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6597b9-4d5a-4a2b-b4f2-57a86e9f9f0b None admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:05.264929 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6597b9-4d5a-4a2b-b4f2-57a86e9f9f0b None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:05.265202 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6597b9-4d5a-4a2b-b4f2-57a86e9f9f0b None admin] PATH_INFO: `/v3/groups` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:05.265983 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6597b9-4d5a-4a2b-b4f2-57a86e9f9f0b None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:05.267935 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6597b9-4d5a-4a2b-b4f2-57a86e9f9f0b None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:05.304596 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f18f7ca-3d58-4a03-8ee5-832e98c730f8 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:05.325299 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:05.340390 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eec49c-714c-46de-835d-82ca75167d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:05.341087 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eec49c-714c-46de-835d-82ca75167d99 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:05.341363 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eec49c-714c-46de-835d-82ca75167d99 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:05.341640 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eec49c-714c-46de-835d-82ca75167d99 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:05.342396 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eec49c-714c-46de-835d-82ca75167d99 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:05.342730 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eec49c-714c-46de-835d-82ca75167d99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:05.343350 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 150/301] 10.222.0.33 () {60 vars in 1134 bytes} [Thu Jun 2 19:00:05 2022] POST /identity/v3/groups => generated 229 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:00:05.352060 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eec49c-714c-46de-835d-82ca75167d99 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:05.365458 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 152/302] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 2 19:00:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:05.372000 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6597b9-4d5a-4a2b-b4f2-57a86e9f9f0b None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:05.402043 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:05.500012 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6597b9-4d5a-4a2b-b4f2-57a86e9f9f0b None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:05.541964 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87770117-309c-4a07-9619-3f0e4aa193e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:05.542911 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87770117-309c-4a07-9619-3f0e4aa193e9 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:05.543200 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87770117-309c-4a07-9619-3f0e4aa193e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:05.543480 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87770117-309c-4a07-9619-3f0e4aa193e9 None admin] PATH_INFO: `/v3/users/af89311ca4174878a8bc5f62dc200ca2` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:05.564301 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87770117-309c-4a07-9619-3f0e4aa193e9 None admin] RBAC: Authorizing `identity:get_user(user_id=af89311ca4174878a8bc5f62dc200ca2)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:05.566155 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87770117-309c-4a07-9619-3f0e4aa193e9 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:05.567646 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 151/303] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 2 19:00:05 2022] GET /identity/v3/users/af89311ca4174878a8bc5f62dc200ca2 => generated 293 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:05.574949 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eec49c-714c-46de-835d-82ca75167d99 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:05.674971 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe58632-393f-42a9-a809-2a188fb0a2a6 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:05.675323 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe58632-393f-42a9-a809-2a188fb0a2a6 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:05.675566 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe58632-393f-42a9-a809-2a188fb0a2a6 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:05.677220 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 152/304] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:00:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:00:05.684501 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46255d8-e59e-4b62-9169-5c9f079680b2 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:05.684914 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46255d8-e59e-4b62-9169-5c9f079680b2 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:05.685294 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46255d8-e59e-4b62-9169-5c9f079680b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:05.696615 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eec49c-714c-46de-835d-82ca75167d99 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:05.707455 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-e46255d8-e59e-4b62-9169-5c9f079680b2 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:00:05.709458 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-e46255d8-e59e-4b62-9169-5c9f079680b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:00:05.733554 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6428e34-86ff-4788-b1b9-5a00338b02e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:05.736708 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6428e34-86ff-4788-b1b9-5a00338b02e7 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:05.736708 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6428e34-86ff-4788-b1b9-5a00338b02e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:05.736708 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6428e34-86ff-4788-b1b9-5a00338b02e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:05.736708 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6428e34-86ff-4788-b1b9-5a00338b02e7 None admin] RBAC: Adding query filter params (role.id=de9eea600632474d8b149397589c3746, user.id=af89311ca4174878a8bc5f62dc200ca2, scope.system=all) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:05.736708 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6428e34-86ff-4788-b1b9-5a00338b02e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:05.738108 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6428e34-86ff-4788-b1b9-5a00338b02e7 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:05.744412 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 153/305] 10.222.0.33 () {56 vars in 1308 bytes} [Thu Jun 2 19:00:05 2022] GET /identity/v3/role_assignments?user.id=af89311ca4174878a8bc5f62dc200ca2&scope.system=all&role.id=de9eea600632474d8b149397589c3746 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:05.801574 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 153/306] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:00:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 19:00:05.824273 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6428e34-86ff-4788-b1b9-5a00338b02e7 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:05.882813 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:05.906729 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8acdf6e-6a5b-46e3-8148-436a6e9a48a8 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:05.907618 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8acdf6e-6a5b-46e3-8148-436a6e9a48a8 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:05.908132 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8acdf6e-6a5b-46e3-8148-436a6e9a48a8 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:05.909795 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 154/307] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:00:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 02 19:00:05.916364 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c22c7-ef73-4b2f-9381-dbb656d4e59f None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:05.916710 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c22c7-ef73-4b2f-9381-dbb656d4e59f None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:05.916925 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c22c7-ef73-4b2f-9381-dbb656d4e59f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:05.938560 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-2d1c22c7-ef73-4b2f-9381-dbb656d4e59f None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:00:05.941024 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-2d1c22c7-ef73-4b2f-9381-dbb656d4e59f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:00:05.944586 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:05.951229 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6428e34-86ff-4788-b1b9-5a00338b02e7 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:05.990507 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2929abe4-d350-467a-a76e-208e11d1100a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:05.992489 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2929abe4-d350-467a-a76e-208e11d1100a None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:05.992780 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2929abe4-d350-467a-a76e-208e11d1100a None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:05.993076 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2929abe4-d350-467a-a76e-208e11d1100a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:06.006290 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2929abe4-d350-467a-a76e-208e11d1100a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:00:06.014424 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-2929abe4-d350-467a-a76e-208e11d1100a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 02 19:00:06.014424 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:06.014424 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:06.014424 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:06.014424 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:06.014424 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:06.014424 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:06.014424 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:06.014424 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:06.014424 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:06.014424 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:06.014424 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:06.014424 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:00:06.014424 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:00:06.014424 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:00:06.014424 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:00:06.015417 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:00:06.015417 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:00:06.015417 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:00:06.015417 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:00:06.015417 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:06.015417 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:06.015417 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:06.015417 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:06.015417 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:06.015417 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:06.015417 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:06.015417 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:06.015417 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:06.015417 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:06.015417 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:06.015417 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:06.016272 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:06.016272 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:06.016272 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:06.016272 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:06.016272 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:00:06.016272 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:00:06.016272 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:00:06.016272 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:00:06.016272 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:00:06.016272 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:00:06.016272 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 02 19:00:06.016272 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:00:06.018739 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 154/308] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 2 19:00:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:06.030942 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2929abe4-d350-467a-a76e-208e11d1100a None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:06.031201 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 155/309] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:00:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 02 19:00:06.060474 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1c22c7-ef73-4b2f-9381-dbb656d4e59f None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:06.154238 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2929abe4-d350-467a-a76e-208e11d1100a None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:06.165092 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:06.185008 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1c22c7-ef73-4b2f-9381-dbb656d4e59f None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:06.201464 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13d2850-df90-4160-9008-9d45c2461307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:06.202621 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13d2850-df90-4160-9008-9d45c2461307 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:06.202948 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13d2850-df90-4160-9008-9d45c2461307 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:06.203234 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13d2850-df90-4160-9008-9d45c2461307 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:06.204276 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13d2850-df90-4160-9008-9d45c2461307 None admin] RBAC: Adding query filter params (name=member) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:06.204591 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13d2850-df90-4160-9008-9d45c2461307 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:06.206663 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13d2850-df90-4160-9008-9d45c2461307 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:06.222787 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 155/310] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 2 19:00:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:00:06.225751 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13d2850-df90-4160-9008-9d45c2461307 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:06.230142 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae462fde-e331-497d-be9a-31cac6c6994c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:06.231091 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae462fde-e331-497d-be9a-31cac6c6994c None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:06.231596 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae462fde-e331-497d-be9a-31cac6c6994c None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:06.235136 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae462fde-e331-497d-be9a-31cac6c6994c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:06.246024 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:06.251421 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae462fde-e331-497d-be9a-31cac6c6994c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:00:06.262592 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-ae462fde-e331-497d-be9a-31cac6c6994c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 02 19:00:06.262592 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:06.262592 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:06.262592 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:06.262592 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:06.262592 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:06.262592 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:06.262592 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:06.262592 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:06.262592 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:06.262592 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:06.262592 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:06.262592 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:00:06.262592 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:00:06.262592 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:00:06.262592 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:00:06.263575 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:00:06.263575 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:00:06.263575 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:00:06.263575 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:00:06.263575 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:06.263575 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:06.263575 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:06.263575 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:06.263575 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:06.263575 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:06.263575 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:06.263575 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:06.263575 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:06.263575 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:06.263575 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:06.263575 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:06.264363 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:06.264363 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:06.264363 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:06.264363 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:06.264363 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:00:06.264363 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:00:06.264363 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:00:06.264363 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:00:06.264363 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:00:06.264363 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:00:06.264363 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 02 19:00:06.264363 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:00:06.266833 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 156/311] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 2 19:00:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 02 19:00:06.269691 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae462fde-e331-497d-be9a-31cac6c6994c None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:06.356785 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13d2850-df90-4160-9008-9d45c2461307 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:06.410798 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae462fde-e331-497d-be9a-31cac6c6994c None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:06.411865 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13422523-4d55-4f02-ad73-48570b3bb56c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:06.414276 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13422523-4d55-4f02-ad73-48570b3bb56c None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:06.415124 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13422523-4d55-4f02-ad73-48570b3bb56c None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:06.415887 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13422523-4d55-4f02-ad73-48570b3bb56c None admin] PATH_INFO: `/v3/users/5d028041b39a4c91ad4022d4b766ada3` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:06.418387 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13422523-4d55-4f02-ad73-48570b3bb56c None admin] RBAC: Authorizing `identity:get_user(user_id=5d028041b39a4c91ad4022d4b766ada3)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:06.421614 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13422523-4d55-4f02-ad73-48570b3bb56c None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:06.430784 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 156/312] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 2 19:00:06 2022] GET /identity/v3/users/5d028041b39a4c91ad4022d4b766ada3 => generated 293 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:00:06.435285 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13422523-4d55-4f02-ad73-48570b3bb56c None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:06.448415 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e97ce76-c69d-4b21-bad2-0ff07155250f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:06.449096 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e97ce76-c69d-4b21-bad2-0ff07155250f None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:06.449388 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e97ce76-c69d-4b21-bad2-0ff07155250f None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:06.449654 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e97ce76-c69d-4b21-bad2-0ff07155250f None admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:06.450414 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e97ce76-c69d-4b21-bad2-0ff07155250f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:06.450725 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e97ce76-c69d-4b21-bad2-0ff07155250f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:06.452153 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e97ce76-c69d-4b21-bad2-0ff07155250f None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:06.466835 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 157/313] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 2 19:00:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:00:06.468993 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e97ce76-c69d-4b21-bad2-0ff07155250f None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:06.494931 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:06.549258 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:06.578297 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13422523-4d55-4f02-ad73-48570b3bb56c None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:06.610918 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e97ce76-c69d-4b21-bad2-0ff07155250f None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:06.621339 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c60aa60-984c-4177-bc56-b1a97496ce92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:06.622825 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c60aa60-984c-4177-bc56-b1a97496ce92 None admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:06.623056 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c60aa60-984c-4177-bc56-b1a97496ce92 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:06.623466 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c60aa60-984c-4177-bc56-b1a97496ce92 None admin] PATH_INFO: `/v3/system/users/5d028041b39a4c91ad4022d4b766ada3/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:06.625479 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c60aa60-984c-4177-bc56-b1a97496ce92 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=5d028041b39a4c91ad4022d4b766ada3, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:06.627546 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c60aa60-984c-4177-bc56-b1a97496ce92 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:06.656510 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ba6e1a-5f69-4183-bb54-0ee9f4412a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:06.657397 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba6e1a-5f69-4183-bb54-0ee9f4412a22 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:06.657678 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba6e1a-5f69-4183-bb54-0ee9f4412a22 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:06.657956 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba6e1a-5f69-4183-bb54-0ee9f4412a22 None admin] PATH_INFO: `/v3/users/dffa20f30b714c7580ce966ceba58add` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:06.659196 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ba6e1a-5f69-4183-bb54-0ee9f4412a22 None admin] RBAC: Authorizing `identity:get_user(user_id=dffa20f30b714c7580ce966ceba58add)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:06.660642 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ba6e1a-5f69-4183-bb54-0ee9f4412a22 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:06.666880 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 158/314] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 2 19:00:06 2022] GET /identity/v3/users/dffa20f30b714c7580ce966ceba58add => generated 297 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:06.668267 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ba6e1a-5f69-4183-bb54-0ee9f4412a22 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:06.715486 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 157/315] 10.222.0.33 () {58 vars in 1334 bytes} [Thu Jun 2 19:00:06 2022] PUT /identity/v3/system/users/5d028041b39a4c91ad4022d4b766ada3/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 02 19:00:06.790778 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ba6e1a-5f69-4183-bb54-0ee9f4412a22 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:06.821741 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:06.833110 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b77180-60af-4d96-bb27-3527549d50d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:06.833834 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b77180-60af-4d96-bb27-3527549d50d4 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:06.834178 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b77180-60af-4d96-bb27-3527549d50d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:06.835014 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b77180-60af-4d96-bb27-3527549d50d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:06.929205 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b77180-60af-4d96-bb27-3527549d50d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:06.931830 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b77180-60af-4d96-bb27-3527549d50d4 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:07.046161 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 159/316] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 2 19:00:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 02 19:00:07.051613 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c60aa60-984c-4177-bc56-b1a97496ce92 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:07.217596 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:07.223138 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c60aa60-984c-4177-bc56-b1a97496ce92 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:07.283854 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a41f7b4-722a-4918-8d18-611140d062c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:07.285813 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a41f7b4-722a-4918-8d18-611140d062c5 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:07.286661 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a41f7b4-722a-4918-8d18-611140d062c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:07.287978 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a41f7b4-722a-4918-8d18-611140d062c5 None admin] PATH_INFO: `/v3/projects/5f19550a1b254991ba8eb40da4841fcf` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:07.290428 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a41f7b4-722a-4918-8d18-611140d062c5 None admin] RBAC: Authorizing `identity:get_project(project_id=5f19550a1b254991ba8eb40da4841fcf)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:07.293469 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a41f7b4-722a-4918-8d18-611140d062c5 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:07.297367 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 158/317] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 2 19:00:07 2022] GET /identity/v3/projects/5f19550a1b254991ba8eb40da4841fcf => generated 302 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:07.306468 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b77180-60af-4d96-bb27-3527549d50d4 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:07.357834 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:07.458090 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b77180-60af-4d96-bb27-3527549d50d4 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:07.500514 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e98a9e9-4daf-4246-afc6-736ff374afff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:07.501592 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e98a9e9-4daf-4246-afc6-736ff374afff None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:07.502340 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e98a9e9-4daf-4246-afc6-736ff374afff None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:07.502874 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e98a9e9-4daf-4246-afc6-736ff374afff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:07.503922 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e98a9e9-4daf-4246-afc6-736ff374afff None admin] RBAC: Adding query filter params (role.id=de9eea600632474d8b149397589c3746, scope.project.id=5f19550a1b254991ba8eb40da4841fcf, user.id=dffa20f30b714c7580ce966ceba58add) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:07.504428 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e98a9e9-4daf-4246-afc6-736ff374afff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:07.506076 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e98a9e9-4daf-4246-afc6-736ff374afff None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:07.513899 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 160/318] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 2 19:00:07 2022] GET /identity/v3/role_assignments?user.id=dffa20f30b714c7580ce966ceba58add&scope.project.id=5f19550a1b254991ba8eb40da4841fcf&role.id=de9eea600632474d8b149397589c3746 => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:07.797886 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ae583f-1290-41da-9c09-aa52fd184577 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:07.798202 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ae583f-1290-41da-9c09-aa52fd184577 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:07.798540 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ae583f-1290-41da-9c09-aa52fd184577 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:07.800787 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 159/319] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:00:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:00:07.807323 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db624c9-bae7-48ad-b37a-49dfdb5cc31b None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:07.807662 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db624c9-bae7-48ad-b37a-49dfdb5cc31b None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:07.807901 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db624c9-bae7-48ad-b37a-49dfdb5cc31b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:07.829160 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-8db624c9-bae7-48ad-b37a-49dfdb5cc31b None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:00:07.830398 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-8db624c9-bae7-48ad-b37a-49dfdb5cc31b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:00:07.905863 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 161/320] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:00:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 19:00:07.928378 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ae583f-1290-41da-9c09-aa52fd184577 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:08.032994 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ae583f-1290-41da-9c09-aa52fd184577 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:08.069473 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15008757-9150-479c-b188-43b0232f85e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:08.070436 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15008757-9150-479c-b188-43b0232f85e5 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:08.070575 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15008757-9150-479c-b188-43b0232f85e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:08.070929 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15008757-9150-479c-b188-43b0232f85e5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:08.072028 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15008757-9150-479c-b188-43b0232f85e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:08.073502 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15008757-9150-479c-b188-43b0232f85e5 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:08.075426 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 160/321] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:00:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:08.081154 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db624c9-bae7-48ad-b37a-49dfdb5cc31b None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:08.109885 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:08.131456 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e087aed-8420-4b75-80a9-5f00ebca3caf None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:08.131803 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e087aed-8420-4b75-80a9-5f00ebca3caf None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:08.132027 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e087aed-8420-4b75-80a9-5f00ebca3caf None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:08.133691 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 161/322] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:00:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:00:08.139626 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f9bfe2-dd21-4606-990e-005ad8cf61d3 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:08.139919 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f9bfe2-dd21-4606-990e-005ad8cf61d3 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:08.140207 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f9bfe2-dd21-4606-990e-005ad8cf61d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:08.157963 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-b7f9bfe2-dd21-4606-990e-005ad8cf61d3 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:00:08.159213 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-b7f9bfe2-dd21-4606-990e-005ad8cf61d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:00:08.193524 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:08.212964 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db624c9-bae7-48ad-b37a-49dfdb5cc31b None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:08.242793 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 162/323] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:00:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 02 19:00:08.251212 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8565ae-8fdf-4b58-a1c5-d0c9b95aba3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:08.251894 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8565ae-8fdf-4b58-a1c5-d0c9b95aba3a None admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:08.252441 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8565ae-8fdf-4b58-a1c5-d0c9b95aba3a None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:08.252720 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8565ae-8fdf-4b58-a1c5-d0c9b95aba3a None admin] PATH_INFO: `/v3/groups` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:08.253430 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8565ae-8fdf-4b58-a1c5-d0c9b95aba3a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:08.254913 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8565ae-8fdf-4b58-a1c5-d0c9b95aba3a None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:08.262565 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f9bfe2-dd21-4606-990e-005ad8cf61d3 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:08.356244 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 162/324] 10.222.0.33 () {60 vars in 1134 bytes} [Thu Jun 2 19:00:08 2022] POST /identity/v3/groups => generated 226 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:00:08.375523 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f9bfe2-dd21-4606-990e-005ad8cf61d3 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:08.416109 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e352a5-5ae4-401f-87d7-a1e7b0de03d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:08.416936 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e352a5-5ae4-401f-87d7-a1e7b0de03d7 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:08.417219 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e352a5-5ae4-401f-87d7-a1e7b0de03d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:08.417494 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e352a5-5ae4-401f-87d7-a1e7b0de03d7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:08.430488 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e352a5-5ae4-401f-87d7-a1e7b0de03d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:00:08.438583 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-70e352a5-5ae4-401f-87d7-a1e7b0de03d7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 02 19:00:08.438583 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:08.438583 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:08.438583 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:08.438583 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:08.438583 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:08.438583 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:08.438583 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:08.438583 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:08.438583 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:08.438583 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:08.438583 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:08.438583 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:00:08.438583 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:00:08.438583 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:00:08.438583 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:00:08.439727 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:00:08.439727 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:00:08.439727 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:00:08.439727 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:00:08.439727 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:08.439727 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:08.439727 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:08.439727 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:08.439727 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:08.439727 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:08.439727 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:08.439727 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:08.439727 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:08.439727 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:08.439727 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:08.439727 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:08.445861 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:08.445861 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:08.445861 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:08.445861 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:08.445861 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:00:08.445861 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:00:08.445861 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:00:08.445861 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:00:08.445861 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:00:08.445861 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:00:08.445861 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 02 19:00:08.445861 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:00:08.445861 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 163/325] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 2 19:00:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 02 19:00:08.448786 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8565ae-8fdf-4b58-a1c5-d0c9b95aba3a None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:08.544547 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:08.565590 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8565ae-8fdf-4b58-a1c5-d0c9b95aba3a None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:08.605859 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80010d69-314d-493f-8e5b-3ead610147ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:08.606942 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80010d69-314d-493f-8e5b-3ead610147ee None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:08.607438 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80010d69-314d-493f-8e5b-3ead610147ee None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:08.608277 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80010d69-314d-493f-8e5b-3ead610147ee None admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:08.609518 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80010d69-314d-493f-8e5b-3ead610147ee None admin] RBAC: Adding query filter params (name=reader) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:08.609518 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80010d69-314d-493f-8e5b-3ead610147ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:08.610740 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80010d69-314d-493f-8e5b-3ead610147ee None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:08.620240 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 163/326] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 2 19:00:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:08.626135 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e352a5-5ae4-401f-87d7-a1e7b0de03d7 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:08.658861 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:08.744503 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e352a5-5ae4-401f-87d7-a1e7b0de03d7 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:08.784359 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd06fd6-f3df-4e42-a513-16140cd71ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:08.785270 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd06fd6-f3df-4e42-a513-16140cd71ab1 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:08.785555 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd06fd6-f3df-4e42-a513-16140cd71ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:08.785828 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd06fd6-f3df-4e42-a513-16140cd71ab1 None admin] PATH_INFO: `/v3/users/af89311ca4174878a8bc5f62dc200ca2` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:08.787216 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd06fd6-f3df-4e42-a513-16140cd71ab1 None admin] RBAC: Authorizing `identity:get_user(user_id=af89311ca4174878a8bc5f62dc200ca2)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:08.788737 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd06fd6-f3df-4e42-a513-16140cd71ab1 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:08.790614 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 164/327] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 2 19:00:08 2022] GET /identity/v3/users/af89311ca4174878a8bc5f62dc200ca2 => generated 293 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:08.796298 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80010d69-314d-493f-8e5b-3ead610147ee None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:08.908403 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80010d69-314d-493f-8e5b-3ead610147ee None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:08.951578 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd7b0e2-6a88-43c2-8b99-a7f85597e49a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:08.953101 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd7b0e2-6a88-43c2-8b99-a7f85597e49a None admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:08.953595 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd7b0e2-6a88-43c2-8b99-a7f85597e49a None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:08.954067 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd7b0e2-6a88-43c2-8b99-a7f85597e49a None admin] PATH_INFO: `/v3/system/users/af89311ca4174878a8bc5f62dc200ca2/roles/de9eea600632474d8b149397589c3746` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:08.957128 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd7b0e2-6a88-43c2-8b99-a7f85597e49a None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=af89311ca4174878a8bc5f62dc200ca2, role_id=de9eea600632474d8b149397589c3746)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:08.960427 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd7b0e2-6a88-43c2-8b99-a7f85597e49a None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:09.042081 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 164/328] 10.222.0.33 () {58 vars in 1334 bytes} [Thu Jun 2 19:00:08 2022] PUT /identity/v3/system/users/af89311ca4174878a8bc5f62dc200ca2/roles/de9eea600632474d8b149397589c3746 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:00:09.312179 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dc2d67-587c-4351-a785-42098c9bdde8 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:09.312689 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dc2d67-587c-4351-a785-42098c9bdde8 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:09.312768 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dc2d67-587c-4351-a785-42098c9bdde8 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:09.314297 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 165/329] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:00:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:00:09.320704 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822d74f0-c6f0-48d5-ae48-abe0290879ab None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:09.321603 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822d74f0-c6f0-48d5-ae48-abe0290879ab None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:09.322092 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822d74f0-c6f0-48d5-ae48-abe0290879ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:09.347160 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-822d74f0-c6f0-48d5-ae48-abe0290879ab None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:00:09.348706 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-822d74f0-c6f0-48d5-ae48-abe0290879ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:00:09.351950 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:09.430836 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 165/330] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:00:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 19:00:09.478504 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dc2d67-587c-4351-a785-42098c9bdde8 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:09.601936 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:09.610418 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dc2d67-587c-4351-a785-42098c9bdde8 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:09.651112 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9988b7-a680-40c7-a90a-71f042e15c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:09.651871 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9988b7-a680-40c7-a90a-71f042e15c95 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:09.652166 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9988b7-a680-40c7-a90a-71f042e15c95 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:09.652462 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9988b7-a680-40c7-a90a-71f042e15c95 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:09.667679 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9988b7-a680-40c7-a90a-71f042e15c95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:00:09.675564 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-af9988b7-a680-40c7-a90a-71f042e15c95 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 02 19:00:09.675564 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:09.675564 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:09.675564 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:09.675564 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:09.675564 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:09.675564 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:09.675564 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:09.675564 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:09.675564 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:09.675564 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:09.675564 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:09.675564 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:00:09.675564 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:00:09.675564 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:00:09.675564 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:00:09.677293 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:00:09.677293 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:00:09.677293 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:00:09.677293 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:00:09.677293 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:09.677293 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:09.677293 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:09.677293 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:09.677293 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:09.677293 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:09.677293 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:09.677293 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:09.677293 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:09.677293 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:09.677293 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:09.677293 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:09.679556 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:09.679556 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:09.679556 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:09.679556 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:09.679556 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:00:09.679556 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:00:09.679556 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:00:09.679556 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:00:09.679556 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:00:09.679556 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:00:09.679556 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 02 19:00:09.679556 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:00:09.679556 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 166/331] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 2 19:00:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:09.685723 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822d74f0-c6f0-48d5-ae48-abe0290879ab None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:09.832880 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822d74f0-c6f0-48d5-ae48-abe0290879ab None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:09.869991 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bc6903-4318-4e9c-8eb5-d19b06cfdce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:09.870571 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc6903-4318-4e9c-8eb5-d19b06cfdce8 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:09.870881 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc6903-4318-4e9c-8eb5-d19b06cfdce8 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:09.871166 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc6903-4318-4e9c-8eb5-d19b06cfdce8 None admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:09.871942 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bc6903-4318-4e9c-8eb5-d19b06cfdce8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:09.872255 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bc6903-4318-4e9c-8eb5-d19b06cfdce8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:09.873659 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bc6903-4318-4e9c-8eb5-d19b06cfdce8 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:09.888002 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 166/332] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 2 19:00:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:09.888731 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:09.893430 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9988b7-a680-40c7-a90a-71f042e15c95 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:10.025582 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9988b7-a680-40c7-a90a-71f042e15c95 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:10.069180 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24496570-0d30-4e56-be23-636e76a46940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:10.070080 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24496570-0d30-4e56-be23-636e76a46940 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:10.070369 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24496570-0d30-4e56-be23-636e76a46940 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:10.070652 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24496570-0d30-4e56-be23-636e76a46940 None admin] PATH_INFO: `/v3/users/5d028041b39a4c91ad4022d4b766ada3` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:10.072007 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24496570-0d30-4e56-be23-636e76a46940 None admin] RBAC: Authorizing `identity:get_user(user_id=5d028041b39a4c91ad4022d4b766ada3)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:10.073506 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24496570-0d30-4e56-be23-636e76a46940 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:10.075498 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 167/333] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 2 19:00:09 2022] GET /identity/v3/users/5d028041b39a4c91ad4022d4b766ada3 => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:10.081001 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bc6903-4318-4e9c-8eb5-d19b06cfdce8 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:10.234188 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bc6903-4318-4e9c-8eb5-d19b06cfdce8 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:10.274258 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2e7d32-a376-460c-924a-0c1b60828dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:10.274976 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e7d32-a376-460c-924a-0c1b60828dcc None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:10.275264 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e7d32-a376-460c-924a-0c1b60828dcc None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:10.275538 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e7d32-a376-460c-924a-0c1b60828dcc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:10.276369 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2e7d32-a376-460c-924a-0c1b60828dcc None admin] RBAC: Adding query filter params (role.id=77d5301ed78146468d4115b512cc70b7, user.id=5d028041b39a4c91ad4022d4b766ada3, scope.system=all) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:10.276679 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2e7d32-a376-460c-924a-0c1b60828dcc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:10.278118 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2e7d32-a376-460c-924a-0c1b60828dcc None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:10.295726 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 167/334] 10.222.0.33 () {56 vars in 1308 bytes} [Thu Jun 2 19:00:10 2022] GET /identity/v3/role_assignments?user.id=5d028041b39a4c91ad4022d4b766ada3&scope.system=all&role.id=77d5301ed78146468d4115b512cc70b7 => generated 510 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:00:12.227752 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774ff03e-ea41-4f04-b68d-b20a407fbdc8 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:12.228838 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774ff03e-ea41-4f04-b68d-b20a407fbdc8 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:12.229470 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774ff03e-ea41-4f04-b68d-b20a407fbdc8 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:12.231483 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 168/335] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:00:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:00:12.237765 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e4c478-0535-4365-b30e-31ace885ba92 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:12.238826 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e4c478-0535-4365-b30e-31ace885ba92 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:12.239320 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e4c478-0535-4365-b30e-31ace885ba92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:12.254985 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ad7e2-b5d3-4555-8703-8207f8028119 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:12.255615 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ad7e2-b5d3-4555-8703-8207f8028119 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:12.256102 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ad7e2-b5d3-4555-8703-8207f8028119 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:12.259072 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 169/336] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:00:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:00:12.264293 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a65d110-5e63-4140-8be6-167dac2459cd None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:12.264856 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a65d110-5e63-4140-8be6-167dac2459cd None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:12.265322 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a65d110-5e63-4140-8be6-167dac2459cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:12.273962 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-02e4c478-0535-4365-b30e-31ace885ba92 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:00:12.280707 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-02e4c478-0535-4365-b30e-31ace885ba92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:00:12.298540 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-9a65d110-5e63-4140-8be6-167dac2459cd None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:00:12.300121 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-9a65d110-5e63-4140-8be6-167dac2459cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:00:12.336894 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:12.403934 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:12.433718 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 168/337] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:00:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 02 19:00:12.450640 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 170/338] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:00:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 19:00:12.454615 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2569ad56-615f-47da-997f-014029c8f009 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:12.455175 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2569ad56-615f-47da-997f-014029c8f009 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:12.455662 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2569ad56-615f-47da-997f-014029c8f009 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:12.457225 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 169/339] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:00:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:00:12.461294 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a65d110-5e63-4140-8be6-167dac2459cd None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:12.471464 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc05bc-760e-46d8-9f39-b1f3ed0dc956 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:12.472061 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc05bc-760e-46d8-9f39-b1f3ed0dc956 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:12.472543 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc05bc-760e-46d8-9f39-b1f3ed0dc956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:12.493660 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-3acc05bc-760e-46d8-9f39-b1f3ed0dc956 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:00:12.495297 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-3acc05bc-760e-46d8-9f39-b1f3ed0dc956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:00:12.608073 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 170/340] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:00:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 02 19:00:12.609355 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acc05bc-760e-46d8-9f39-b1f3ed0dc956 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:12.656137 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a65d110-5e63-4140-8be6-167dac2459cd None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:12.729256 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af7a89c-87a6-4ff9-a054-17682ac7c296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:12.730140 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af7a89c-87a6-4ff9-a054-17682ac7c296 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:12.730425 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af7a89c-87a6-4ff9-a054-17682ac7c296 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:12.732984 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af7a89c-87a6-4ff9-a054-17682ac7c296 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:12.751369 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af7a89c-87a6-4ff9-a054-17682ac7c296 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:00:12.768936 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-6af7a89c-87a6-4ff9-a054-17682ac7c296 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 02 19:00:12.768936 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:12.768936 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:12.768936 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:12.768936 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:12.768936 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:12.768936 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:12.768936 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:12.768936 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:12.768936 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:12.768936 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:12.768936 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:12.768936 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:00:12.768936 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:00:12.768936 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:00:12.768936 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:00:12.769894 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:00:12.769894 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:00:12.769894 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:00:12.769894 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:00:12.769894 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:12.769894 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:12.769894 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:12.769894 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:12.769894 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:12.769894 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:12.769894 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:12.769894 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:12.769894 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:12.769894 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:12.769894 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:12.769894 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:12.770696 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:12.770696 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:12.770696 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:12.770696 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:12.770696 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:00:12.770696 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:00:12.770696 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:00:12.770696 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:00:12.770696 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:00:12.770696 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:00:12.770696 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 02 19:00:12.770696 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:00:12.772058 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 171/341] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 2 19:00:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:12.772538 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af7a89c-87a6-4ff9-a054-17682ac7c296 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:12.800068 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:12.811320 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acc05bc-760e-46d8-9f39-b1f3ed0dc956 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:12.859131 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:12.877649 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f3dfa2-8398-4d43-a0ac-389f55290165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:12.878464 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f3dfa2-8398-4d43-a0ac-389f55290165 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:12.878767 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f3dfa2-8398-4d43-a0ac-389f55290165 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:12.879044 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f3dfa2-8398-4d43-a0ac-389f55290165 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:12.892666 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f3dfa2-8398-4d43-a0ac-389f55290165 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:00:12.903178 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-81f3dfa2-8398-4d43-a0ac-389f55290165 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 02 19:00:12.903178 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:12.903178 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:12.903178 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:12.903178 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:12.903178 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:12.903178 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:12.903178 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:12.903178 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:12.903178 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:12.903178 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:12.903178 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:12.903178 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:00:12.903178 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:00:12.903178 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:00:12.903178 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:00:12.906237 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:00:12.906237 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:00:12.906237 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:00:12.906237 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:00:12.906237 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:12.906237 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:12.906237 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:12.906237 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:12.906237 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:12.906237 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:12.906237 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:12.906237 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:12.906237 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:12.906237 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:12.906237 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:12.906237 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:12.907380 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:12.907380 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:12.907380 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:12.907380 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:12.907380 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:00:12.907380 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:00:12.907380 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:00:12.907380 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:00:12.907380 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:00:12.907380 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:00:12.907380 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 02 19:00:12.907380 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:00:12.909190 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 171/342] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 2 19:00:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:12.910079 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f3dfa2-8398-4d43-a0ac-389f55290165 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:12.934016 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af7a89c-87a6-4ff9-a054-17682ac7c296 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:12.978130 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1dd4a8-a9c5-4f6d-896f-78d67977d689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:12.978988 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1dd4a8-a9c5-4f6d-896f-78d67977d689 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:12.979279 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1dd4a8-a9c5-4f6d-896f-78d67977d689 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:12.979563 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1dd4a8-a9c5-4f6d-896f-78d67977d689 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:12.993557 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1dd4a8-a9c5-4f6d-896f-78d67977d689 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:00:13.004060 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-4d1dd4a8-a9c5-4f6d-896f-78d67977d689 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 02 19:00:13.004060 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:13.004060 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:13.004060 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:13.004060 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:13.004060 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:13.004060 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:13.004060 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:13.004060 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:13.004060 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:13.004060 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:13.004060 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:13.004060 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:00:13.004060 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:00:13.004060 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:00:13.004060 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:00:13.005024 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:00:13.005024 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:00:13.005024 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:00:13.005024 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:00:13.005024 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:13.005024 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:13.005024 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:13.005024 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:13.005024 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:13.005024 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:13.005024 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:13.005024 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:13.005024 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:13.005024 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:13.005024 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:13.005024 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:13.006266 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:13.006266 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:13.006266 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:13.006266 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:13.006266 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:00:13.006266 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:00:13.006266 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:00:13.006266 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:00:13.006266 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:00:13.006266 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:00:13.006266 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 02 19:00:13.006266 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:00:13.006266 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 172/343] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 2 19:00:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:13.007839 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166e2f4-6311-4001-9047-4efa9429bf6b None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:13.008016 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166e2f4-6311-4001-9047-4efa9429bf6b None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:13.008298 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166e2f4-6311-4001-9047-4efa9429bf6b None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:13.009641 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 173/344] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:00:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:00:13.014489 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f166e2f4-6311-4001-9047-4efa9429bf6b None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:13.056273 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f3dfa2-8398-4d43-a0ac-389f55290165 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:13.102430 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a612f5-7780-4942-8a1c-699a6619d64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:13.103167 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a612f5-7780-4942-8a1c-699a6619d64e None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:13.103459 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a612f5-7780-4942-8a1c-699a6619d64e None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:13.103741 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a612f5-7780-4942-8a1c-699a6619d64e None admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:13.104517 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a612f5-7780-4942-8a1c-699a6619d64e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:13.104844 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a612f5-7780-4942-8a1c-699a6619d64e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:13.106740 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a612f5-7780-4942-8a1c-699a6619d64e None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:13.110855 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:13.121642 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 172/345] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 2 19:00:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:13.122482 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a612f5-7780-4942-8a1c-699a6619d64e None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:13.152165 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f166e2f4-6311-4001-9047-4efa9429bf6b None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:13.197247 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffb2959-b455-4f3a-b448-dd324be516d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:13.198003 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb2959-b455-4f3a-b448-dd324be516d3 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:13.198611 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb2959-b455-4f3a-b448-dd324be516d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:13.199036 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb2959-b455-4f3a-b448-dd324be516d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:13.199942 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffb2959-b455-4f3a-b448-dd324be516d3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:13.200435 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffb2959-b455-4f3a-b448-dd324be516d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:13.202427 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffb2959-b455-4f3a-b448-dd324be516d3 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:13.211236 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:13.215984 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 174/346] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 2 19:00:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:13.217360 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e1ed7e-4aad-4be4-87c8-00353ebe9ea1 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:13.217657 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e1ed7e-4aad-4be4-87c8-00353ebe9ea1 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:13.217942 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e1ed7e-4aad-4be4-87c8-00353ebe9ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:13.247087 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-e3e1ed7e-4aad-4be4-87c8-00353ebe9ea1 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:00:13.248597 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-e3e1ed7e-4aad-4be4-87c8-00353ebe9ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:00:13.258936 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a612f5-7780-4942-8a1c-699a6619d64e None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:13.307178 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c28551c-3232-4326-b943-64d41ddf0119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:13.307863 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c28551c-3232-4326-b943-64d41ddf0119 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:13.308152 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c28551c-3232-4326-b943-64d41ddf0119 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:13.308538 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c28551c-3232-4326-b943-64d41ddf0119 None admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:13.309539 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c28551c-3232-4326-b943-64d41ddf0119 None admin] RBAC: Adding query filter params (name=member) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:13.309857 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c28551c-3232-4326-b943-64d41ddf0119 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:13.311362 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c28551c-3232-4326-b943-64d41ddf0119 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:13.323015 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 173/347] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 2 19:00:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:13.323945 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c28551c-3232-4326-b943-64d41ddf0119 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:13.332883 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 175/348] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:00:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 19:00:13.333761 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e1ed7e-4aad-4be4-87c8-00353ebe9ea1 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:13.411445 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:13.470167 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e1ed7e-4aad-4be4-87c8-00353ebe9ea1 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:13.476767 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c28551c-3232-4326-b943-64d41ddf0119 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:13.478653 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:13.516759 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f09caec-cad6-49af-a862-5df40477fd26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:13.517559 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f09caec-cad6-49af-a862-5df40477fd26 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:13.518487 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f09caec-cad6-49af-a862-5df40477fd26 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:13.518487 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f09caec-cad6-49af-a862-5df40477fd26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:13.522790 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d51b03-8b51-4f42-ae78-453e5984f2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:13.523452 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d51b03-8b51-4f42-ae78-453e5984f2b5 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:13.523738 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d51b03-8b51-4f42-ae78-453e5984f2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:13.525635 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d51b03-8b51-4f42-ae78-453e5984f2b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:13.613249 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d51b03-8b51-4f42-ae78-453e5984f2b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:13.613249 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d51b03-8b51-4f42-ae78-453e5984f2b5 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:13.613744 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f09caec-cad6-49af-a862-5df40477fd26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:13.616514 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f09caec-cad6-49af-a862-5df40477fd26 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:13.702310 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:13.745414 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 174/349] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 2 19:00:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 02 19:00:13.745414 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d51b03-8b51-4f42-ae78-453e5984f2b5 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:13.749525 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:13.765187 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 176/350] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 2 19:00:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 02 19:00:13.766392 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f09caec-cad6-49af-a862-5df40477fd26 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:13.872397 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d51b03-8b51-4f42-ae78-453e5984f2b5 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:13.888280 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f09caec-cad6-49af-a862-5df40477fd26 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:13.911384 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87706dfa-c613-4c14-9f6f-9d44e8186cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:13.912040 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87706dfa-c613-4c14-9f6f-9d44e8186cd2 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:13.912323 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87706dfa-c613-4c14-9f6f-9d44e8186cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:13.912595 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87706dfa-c613-4c14-9f6f-9d44e8186cd2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:13.927496 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7f0bc6-890b-45b9-b80b-4e05eba63c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:13.928398 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f0bc6-890b-45b9-b80b-4e05eba63c5f None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:13.929147 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f0bc6-890b-45b9-b80b-4e05eba63c5f None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:13.929421 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f0bc6-890b-45b9-b80b-4e05eba63c5f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:13.943504 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7f0bc6-890b-45b9-b80b-4e05eba63c5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:00:13.951478 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-ec7f0bc6-890b-45b9-b80b-4e05eba63c5f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 02 19:00:13.951478 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:13.951478 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:13.951478 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:13.951478 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:13.951478 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:13.951478 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:13.951478 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:13.951478 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:13.951478 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:13.951478 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:13.951478 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:13.951478 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:00:13.951478 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:00:13.951478 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:00:13.951478 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:00:13.952391 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:00:13.952391 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:00:13.952391 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:00:13.952391 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:00:13.952391 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:13.952391 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:13.952391 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:13.952391 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:13.952391 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:13.952391 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:13.952391 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:13.952391 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:13.952391 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:13.952391 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:13.952391 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:13.952391 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:13.954259 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:13.954259 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:13.954259 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:13.954259 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:13.954259 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:00:13.954259 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:00:13.954259 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:00:13.954259 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:00:13.954259 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:00:13.954259 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:00:13.954259 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 02 19:00:13.954259 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:00:13.954259 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 177/351] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 2 19:00:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:13.954259 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a327d096-86b5-4dc8-8be8-af954cb88236 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:13.955061 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a327d096-86b5-4dc8-8be8-af954cb88236 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:13.955061 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a327d096-86b5-4dc8-8be8-af954cb88236 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:13.957111 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 178/352] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:00:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:00:13.957111 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a327d096-86b5-4dc8-8be8-af954cb88236 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:13.983839 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:14.002136 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87706dfa-c613-4c14-9f6f-9d44e8186cd2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:14.003872 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87706dfa-c613-4c14-9f6f-9d44e8186cd2 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:14.005985 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:14.083186 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a327d096-86b5-4dc8-8be8-af954cb88236 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:14.125259 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 175/353] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 2 19:00:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 02 19:00:14.126274 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87706dfa-c613-4c14-9f6f-9d44e8186cd2 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:14.132828 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febf8bb9-f1fe-40bc-8780-5daf5dddeb45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:14.133291 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf8bb9-f1fe-40bc-8780-5daf5dddeb45 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:14.134786 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf8bb9-f1fe-40bc-8780-5daf5dddeb45 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:14.134786 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf8bb9-f1fe-40bc-8780-5daf5dddeb45 None admin] PATH_INFO: `/v3/projects/abffdc3e7a9447068ceac783b7558c81` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:14.135790 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febf8bb9-f1fe-40bc-8780-5daf5dddeb45 None admin] RBAC: Authorizing `identity:get_project(project_id=abffdc3e7a9447068ceac783b7558c81)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:14.137260 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febf8bb9-f1fe-40bc-8780-5daf5dddeb45 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:14.139175 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 179/354] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 2 19:00:13 2022] GET /identity/v3/projects/abffdc3e7a9447068ceac783b7558c81 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:14.140512 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1303371-f506-4b4d-8959-802ad41a3f5d None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:14.140512 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1303371-f506-4b4d-8959-802ad41a3f5d None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:14.140775 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1303371-f506-4b4d-8959-802ad41a3f5d None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:14.142791 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 180/355] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:00:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:00:14.143433 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca7a81-aed9-4a71-9a48-f14e4bdcc17f None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:14.143726 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca7a81-aed9-4a71-9a48-f14e4bdcc17f None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:14.144058 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca7a81-aed9-4a71-9a48-f14e4bdcc17f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:14.164450 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-fdca7a81-aed9-4a71-9a48-f14e4bdcc17f None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:00:14.165652 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-fdca7a81-aed9-4a71-9a48-f14e4bdcc17f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:00:14.243161 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 181/356] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:00:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 19:00:14.243868 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdca7a81-aed9-4a71-9a48-f14e4bdcc17f None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:14.252909 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87706dfa-c613-4c14-9f6f-9d44e8186cd2 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:14.257100 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:14.277145 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:14.295414 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8a3b5e-b9f7-491f-a27c-52ee8c73acd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:14.296847 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a3b5e-b9f7-491f-a27c-52ee8c73acd3 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:14.297126 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a3b5e-b9f7-491f-a27c-52ee8c73acd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:14.297399 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a3b5e-b9f7-491f-a27c-52ee8c73acd3 None admin] PATH_INFO: `/v3/projects/abffdc3e7a9447068ceac783b7558c81` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:14.300300 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8a3b5e-b9f7-491f-a27c-52ee8c73acd3 None admin] RBAC: Authorizing `identity:get_project(project_id=abffdc3e7a9447068ceac783b7558c81)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:14.300300 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8a3b5e-b9f7-491f-a27c-52ee8c73acd3 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:14.301864 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 176/357] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 2 19:00:14 2022] GET /identity/v3/projects/abffdc3e7a9447068ceac783b7558c81 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:14.302634 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8a3b5e-b9f7-491f-a27c-52ee8c73acd3 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:14.371611 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdca7a81-aed9-4a71-9a48-f14e4bdcc17f None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:14.418833 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f0a120-64fc-4750-9702-ef2fb31922f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:14.418833 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f0a120-64fc-4750-9702-ef2fb31922f1 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:14.419093 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f0a120-64fc-4750-9702-ef2fb31922f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:14.419307 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f0a120-64fc-4750-9702-ef2fb31922f1 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:14.419945 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f0a120-64fc-4750-9702-ef2fb31922f1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:14.420253 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f0a120-64fc-4750-9702-ef2fb31922f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:14.421651 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f0a120-64fc-4750-9702-ef2fb31922f1 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:14.432113 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 182/358] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 2 19:00:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:14.432973 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f0a120-64fc-4750-9702-ef2fb31922f1 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:14.433488 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8a3b5e-b9f7-491f-a27c-52ee8c73acd3 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:14.481108 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab21bb0-abf3-4fa6-895a-d9b2de7706c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:14.481979 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab21bb0-abf3-4fa6-895a-d9b2de7706c8 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:14.482297 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab21bb0-abf3-4fa6-895a-d9b2de7706c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:14.482556 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab21bb0-abf3-4fa6-895a-d9b2de7706c8 None admin] PATH_INFO: `/v3/projects/5f19550a1b254991ba8eb40da4841fcf` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:14.483901 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab21bb0-abf3-4fa6-895a-d9b2de7706c8 None admin] RBAC: Authorizing `identity:get_project(project_id=5f19550a1b254991ba8eb40da4841fcf)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:14.486329 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab21bb0-abf3-4fa6-895a-d9b2de7706c8 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:14.487129 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 177/359] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 2 19:00:14 2022] GET /identity/v3/projects/5f19550a1b254991ba8eb40da4841fcf => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:14.488312 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef21fd-b1ca-40e9-8785-517bdc35419a None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:14.488600 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef21fd-b1ca-40e9-8785-517bdc35419a None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:14.488925 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef21fd-b1ca-40e9-8785-517bdc35419a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:14.506443 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-eaef21fd-b1ca-40e9-8785-517bdc35419a None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:00:14.508223 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-eaef21fd-b1ca-40e9-8785-517bdc35419a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:00:14.536972 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:14.540586 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:14.558199 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f0a120-64fc-4750-9702-ef2fb31922f1 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:14.581875 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 178/360] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:00:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 19:00:14.583066 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaef21fd-b1ca-40e9-8785-517bdc35419a None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:14.596244 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44925d74-d0bb-4a1a-bd13-d2131ec07c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:14.597120 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44925d74-d0bb-4a1a-bd13-d2131ec07c13 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:14.597413 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44925d74-d0bb-4a1a-bd13-d2131ec07c13 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:14.597692 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44925d74-d0bb-4a1a-bd13-d2131ec07c13 None admin] PATH_INFO: `/v3/groups/37365d258edc4d9b852736f631e192fe` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:14.607669 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44925d74-d0bb-4a1a-bd13-d2131ec07c13 None admin] RBAC: Authorizing `identity:get_group(group_id=37365d258edc4d9b852736f631e192fe)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:14.608451 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44925d74-d0bb-4a1a-bd13-d2131ec07c13 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:14.613858 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 183/361] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Jun 2 19:00:14 2022] GET /identity/v3/groups/37365d258edc4d9b852736f631e192fe => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:14.613858 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44925d74-d0bb-4a1a-bd13-d2131ec07c13 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:14.709218 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaef21fd-b1ca-40e9-8785-517bdc35419a None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:14.730821 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44925d74-d0bb-4a1a-bd13-d2131ec07c13 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:14.753970 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc9db68-c38b-48d4-a21c-94d7f1f81ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:14.754900 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9db68-c38b-48d4-a21c-94d7f1f81ed4 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:14.755193 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9db68-c38b-48d4-a21c-94d7f1f81ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:14.755470 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9db68-c38b-48d4-a21c-94d7f1f81ed4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:14.769765 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc9db68-c38b-48d4-a21c-94d7f1f81ed4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:00:14.773941 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c43b384-b532-4338-8858-b5349193da2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:14.774920 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c43b384-b532-4338-8858-b5349193da2d None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:14.775216 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c43b384-b532-4338-8858-b5349193da2d None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:14.775496 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c43b384-b532-4338-8858-b5349193da2d None admin] PATH_INFO: `/v3/groups/37365d258edc4d9b852736f631e192fe` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:14.776670 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c43b384-b532-4338-8858-b5349193da2d None admin] RBAC: Authorizing `identity:get_group(group_id=37365d258edc4d9b852736f631e192fe)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:14.777970 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-2bc9db68-c38b-48d4-a21c-94d7f1f81ed4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 02 19:00:14.777970 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:14.777970 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:14.777970 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:14.777970 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:14.777970 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:14.777970 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:14.777970 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:14.777970 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:14.777970 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:14.777970 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:14.777970 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:14.777970 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:00:14.777970 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:00:14.777970 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:00:14.777970 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:00:14.778855 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:00:14.778855 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:00:14.778855 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:00:14.778855 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:00:14.778855 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:14.778855 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:14.778855 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:14.778855 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:14.778855 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:14.778855 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:14.778855 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:14.778855 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:14.778855 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:14.778855 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:14.778855 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:14.778855 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:14.779680 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:14.779680 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:14.779680 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:14.779680 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:14.779680 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:00:14.779680 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:00:14.779680 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:00:14.779680 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:00:14.779680 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:00:14.779680 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:00:14.779680 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 02 19:00:14.779680 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:00:14.780306 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c43b384-b532-4338-8858-b5349193da2d None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:14.780374 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 179/362] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 2 19:00:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:14.780446 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc9db68-c38b-48d4-a21c-94d7f1f81ed4 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:14.784558 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 184/363] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Jun 2 19:00:14 2022] GET /identity/v3/groups/37365d258edc4d9b852736f631e192fe => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:14.785379 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c43b384-b532-4338-8858-b5349193da2d None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:14.805004 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:14.810823 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:14.883066 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc9db68-c38b-48d4-a21c-94d7f1f81ed4 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:14.889061 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c43b384-b532-4338-8858-b5349193da2d None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:14.916694 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5897997a-509e-4e27-811c-95535ca9b69f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:14.917564 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5897997a-509e-4e27-811c-95535ca9b69f None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:14.917844 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5897997a-509e-4e27-811c-95535ca9b69f None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:14.918120 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5897997a-509e-4e27-811c-95535ca9b69f None admin] PATH_INFO: `/v3/users/af89311ca4174878a8bc5f62dc200ca2` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:14.919330 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5897997a-509e-4e27-811c-95535ca9b69f None admin] RBAC: Authorizing `identity:get_user(user_id=af89311ca4174878a8bc5f62dc200ca2)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:14.920764 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5897997a-509e-4e27-811c-95535ca9b69f None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:14.922371 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 180/364] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 2 19:00:14 2022] GET /identity/v3/users/af89311ca4174878a8bc5f62dc200ca2 => generated 293 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:14.923150 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5897997a-509e-4e27-811c-95535ca9b69f None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:14.923531 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b35a7c7-5916-4967-b720-3f92dd6b7919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:14.924331 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b35a7c7-5916-4967-b720-3f92dd6b7919 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:14.924604 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b35a7c7-5916-4967-b720-3f92dd6b7919 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:14.924879 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b35a7c7-5916-4967-b720-3f92dd6b7919 None admin] PATH_INFO: `/v3/groups/37365d258edc4d9b852736f631e192fe` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:14.926069 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b35a7c7-5916-4967-b720-3f92dd6b7919 None admin] RBAC: Authorizing `identity:get_group(group_id=37365d258edc4d9b852736f631e192fe)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:14.927543 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b35a7c7-5916-4967-b720-3f92dd6b7919 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:14.929411 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 185/365] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Jun 2 19:00:14 2022] GET /identity/v3/groups/37365d258edc4d9b852736f631e192fe => generated 226 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:14.930040 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b35a7c7-5916-4967-b720-3f92dd6b7919 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:15.014186 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:15.017652 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:15.031470 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5897997a-509e-4e27-811c-95535ca9b69f None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:15.031615 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b35a7c7-5916-4967-b720-3f92dd6b7919 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:15.056543 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d82cc43-da99-4924-bde7-a6f209e8af84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:15.057341 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d82cc43-da99-4924-bde7-a6f209e8af84 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:15.057627 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d82cc43-da99-4924-bde7-a6f209e8af84 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:15.057902 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d82cc43-da99-4924-bde7-a6f209e8af84 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:15.062103 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cfec6a-37da-4ccb-ba26-79bbe4c6dc85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:15.062772 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cfec6a-37da-4ccb-ba26-79bbe4c6dc85 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:15.063070 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cfec6a-37da-4ccb-ba26-79bbe4c6dc85 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:15.063358 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cfec6a-37da-4ccb-ba26-79bbe4c6dc85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:15.064184 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cfec6a-37da-4ccb-ba26-79bbe4c6dc85 None admin] RBAC: Adding query filter params (group.id=37365d258edc4d9b852736f631e192fe, role.id=2e0c054b646e4ed2a0e7e5962f09a50e, scope.project.id=abffdc3e7a9447068ceac783b7558c81) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:15.064511 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cfec6a-37da-4ccb-ba26-79bbe4c6dc85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:15.065944 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cfec6a-37da-4ccb-ba26-79bbe4c6dc85 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:15.068053 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d82cc43-da99-4924-bde7-a6f209e8af84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:00:15.074574 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 186/366] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Jun 2 19:00:14 2022] GET /identity/v3/role_assignments?group.id=37365d258edc4d9b852736f631e192fe&scope.project.id=abffdc3e7a9447068ceac783b7558c81&role.id=2e0c054b646e4ed2a0e7e5962f09a50e => generated 260 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:15.075409 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cfec6a-37da-4ccb-ba26-79bbe4c6dc85 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:15.076760 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-7d82cc43-da99-4924-bde7-a6f209e8af84 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 02 19:00:15.076760 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:15.076760 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:15.076760 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:15.076760 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:15.076760 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:15.076760 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:15.076760 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:15.076760 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:15.076760 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:15.076760 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:15.076760 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:15.076760 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:00:15.076760 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:00:15.076760 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:00:15.076760 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:00:15.077804 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:00:15.077804 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:00:15.077804 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:00:15.077804 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:00:15.077804 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:15.077804 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:15.077804 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:15.077804 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:15.077804 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:15.077804 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:15.077804 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:15.077804 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:15.077804 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:15.077804 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:15.077804 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:15.077804 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:15.078600 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:15.078600 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:15.078600 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:15.078600 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:15.078600 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:00:15.078600 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:00:15.078600 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:00:15.078600 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:00:15.078600 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:00:15.078600 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:00:15.078600 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 02 19:00:15.078600 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:00:15.083398 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 181/367] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 2 19:00:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:15.084485 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d82cc43-da99-4924-bde7-a6f209e8af84 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:15.206735 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cfec6a-37da-4ccb-ba26-79bbe4c6dc85 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:15.211291 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d82cc43-da99-4924-bde7-a6f209e8af84 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:15.249072 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ccb7ed-8cbe-44dd-af2e-c9f72b7a09e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:15.249793 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ccb7ed-8cbe-44dd-af2e-c9f72b7a09e2 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:15.250102 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ccb7ed-8cbe-44dd-af2e-c9f72b7a09e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:15.250435 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ccb7ed-8cbe-44dd-af2e-c9f72b7a09e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:15.250719 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e7aecf-c0bd-414d-a484-425960ac8a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:15.251243 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ccb7ed-8cbe-44dd-af2e-c9f72b7a09e2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:15.251431 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e7aecf-c0bd-414d-a484-425960ac8a4a None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:15.251580 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ccb7ed-8cbe-44dd-af2e-c9f72b7a09e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:15.251724 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e7aecf-c0bd-414d-a484-425960ac8a4a None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:15.251982 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e7aecf-c0bd-414d-a484-425960ac8a4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:15.253102 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e7aecf-c0bd-414d-a484-425960ac8a4a None admin] RBAC: Adding query filter params (group.id=37365d258edc4d9b852736f631e192fe, role.id=77d5301ed78146468d4115b512cc70b7, scope.project.id=abffdc3e7a9447068ceac783b7558c81) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:15.253271 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ccb7ed-8cbe-44dd-af2e-c9f72b7a09e2 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:15.253416 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e7aecf-c0bd-414d-a484-425960ac8a4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:15.254922 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e7aecf-c0bd-414d-a484-425960ac8a4a None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:15.261808 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 182/368] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Jun 2 19:00:15 2022] GET /identity/v3/role_assignments?group.id=37365d258edc4d9b852736f631e192fe&scope.project.id=abffdc3e7a9447068ceac783b7558c81&role.id=77d5301ed78146468d4115b512cc70b7 => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:15.262667 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e7aecf-c0bd-414d-a484-425960ac8a4a None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:15.264072 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 187/369] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 2 19:00:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:15.265144 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ccb7ed-8cbe-44dd-af2e-c9f72b7a09e2 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:15.285167 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:15.308012 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:15.411178 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ccb7ed-8cbe-44dd-af2e-c9f72b7a09e2 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:15.416583 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e7aecf-c0bd-414d-a484-425960ac8a4a None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:15.457053 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d849c8f-a971-4753-8245-45ce7f4e1a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:15.457993 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d849c8f-a971-4753-8245-45ce7f4e1a8a None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:15.458427 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d849c8f-a971-4753-8245-45ce7f4e1a8a None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:15.458897 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d849c8f-a971-4753-8245-45ce7f4e1a8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:15.460447 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d849c8f-a971-4753-8245-45ce7f4e1a8a None admin] RBAC: Adding query filter params (group.id=37365d258edc4d9b852736f631e192fe, role.id=77d5301ed78146468d4115b512cc70b7, scope.project.id=5f19550a1b254991ba8eb40da4841fcf) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:15.460940 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d849c8f-a971-4753-8245-45ce7f4e1a8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:15.461421 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc375ac-1059-4992-a1ca-67658cef25b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:15.462090 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc375ac-1059-4992-a1ca-67658cef25b3 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:15.462379 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc375ac-1059-4992-a1ca-67658cef25b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:15.462660 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc375ac-1059-4992-a1ca-67658cef25b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:15.463411 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d849c8f-a971-4753-8245-45ce7f4e1a8a None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:15.463524 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc375ac-1059-4992-a1ca-67658cef25b3 None admin] RBAC: Adding query filter params (role.id=de9eea600632474d8b149397589c3746, user.id=af89311ca4174878a8bc5f62dc200ca2, scope.system=all) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:15.463842 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc375ac-1059-4992-a1ca-67658cef25b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:15.465276 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc375ac-1059-4992-a1ca-67658cef25b3 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:15.470622 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 188/370] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Jun 2 19:00:15 2022] GET /identity/v3/role_assignments?group.id=37365d258edc4d9b852736f631e192fe&scope.project.id=5f19550a1b254991ba8eb40da4841fcf&role.id=77d5301ed78146468d4115b512cc70b7 => generated 260 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:15.471416 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d849c8f-a971-4753-8245-45ce7f4e1a8a None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:15.490986 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 183/371] 10.222.0.33 () {56 vars in 1308 bytes} [Thu Jun 2 19:00:15 2022] GET /identity/v3/role_assignments?user.id=af89311ca4174878a8bc5f62dc200ca2&scope.system=all&role.id=de9eea600632474d8b149397589c3746 => generated 510 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:15.491912 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc375ac-1059-4992-a1ca-67658cef25b3 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:15.629790 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:15.679650 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d849c8f-a971-4753-8245-45ce7f4e1a8a None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:15.693777 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:15.746891 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc375ac-1059-4992-a1ca-67658cef25b3 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:15.747939 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8fd636-49f9-4833-95c1-21362e02f7c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:15.748822 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8fd636-49f9-4833-95c1-21362e02f7c8 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:15.749115 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8fd636-49f9-4833-95c1-21362e02f7c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:15.749385 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8fd636-49f9-4833-95c1-21362e02f7c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:15.750185 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8fd636-49f9-4833-95c1-21362e02f7c8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:15.750497 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8fd636-49f9-4833-95c1-21362e02f7c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:15.753253 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8fd636-49f9-4833-95c1-21362e02f7c8 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:15.771644 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 189/372] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 2 19:00:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:15.772821 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8fd636-49f9-4833-95c1-21362e02f7c8 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:15.844619 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b431e1-f8c5-4e31-b945-c6fea274d2e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:15.845297 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b431e1-f8c5-4e31-b945-c6fea274d2e7 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:15.845582 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b431e1-f8c5-4e31-b945-c6fea274d2e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:15.845864 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b431e1-f8c5-4e31-b945-c6fea274d2e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:15.940695 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b431e1-f8c5-4e31-b945-c6fea274d2e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:15.942376 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b431e1-f8c5-4e31-b945-c6fea274d2e7 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:15.957753 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8fd636-49f9-4833-95c1-21362e02f7c8 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:15.995421 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b72d1-8afb-4159-b502-135586a0f99f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:15.996078 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b72d1-8afb-4159-b502-135586a0f99f None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:15.996369 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b72d1-8afb-4159-b502-135586a0f99f None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:15.996653 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b72d1-8afb-4159-b502-135586a0f99f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:16.000367 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:16.073420 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2b72d1-8afb-4159-b502-135586a0f99f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:16.074947 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2b72d1-8afb-4159-b502-135586a0f99f None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:16.098578 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:16.133226 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 184/373] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 2 19:00:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 642 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 02 19:00:16.139191 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b431e1-f8c5-4e31-b945-c6fea274d2e7 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:16.188793 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 190/374] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 2 19:00:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 02 19:00:16.194996 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b72d1-8afb-4159-b502-135586a0f99f None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:16.266491 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:16.335803 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b431e1-f8c5-4e31-b945-c6fea274d2e7 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:16.337427 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b72d1-8afb-4159-b502-135586a0f99f None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:16.373924 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77033ab-e1fd-4fcb-9cc8-e0b51cd93acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:16.374871 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77033ab-e1fd-4fcb-9cc8-e0b51cd93acf None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:16.375159 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77033ab-e1fd-4fcb-9cc8-e0b51cd93acf None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:16.375436 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77033ab-e1fd-4fcb-9cc8-e0b51cd93acf None admin] PATH_INFO: `/v3/projects/879ee62507da4a8282b2418099da56e5` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:16.376619 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77033ab-e1fd-4fcb-9cc8-e0b51cd93acf None admin] RBAC: Authorizing `identity:get_project(project_id=879ee62507da4a8282b2418099da56e5)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:16.378324 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77033ab-e1fd-4fcb-9cc8-e0b51cd93acf None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:16.380365 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 191/375] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 2 19:00:16 2022] GET /identity/v3/projects/879ee62507da4a8282b2418099da56e5 => generated 344 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:00:16.386212 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77033ab-e1fd-4fcb-9cc8-e0b51cd93acf None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:16.399885 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e98664-8909-41e5-98ff-eb5cfa2a56b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:16.401301 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e98664-8909-41e5-98ff-eb5cfa2a56b5 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:16.401784 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e98664-8909-41e5-98ff-eb5cfa2a56b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:16.402281 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e98664-8909-41e5-98ff-eb5cfa2a56b5 None admin] PATH_INFO: `/v3/projects/5f19550a1b254991ba8eb40da4841fcf` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:16.403986 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e98664-8909-41e5-98ff-eb5cfa2a56b5 None admin] RBAC: Authorizing `identity:get_project(project_id=5f19550a1b254991ba8eb40da4841fcf)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:16.405727 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e98664-8909-41e5-98ff-eb5cfa2a56b5 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:16.407948 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 185/376] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 2 19:00:16 2022] GET /identity/v3/projects/5f19550a1b254991ba8eb40da4841fcf => generated 302 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:16.414557 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e98664-8909-41e5-98ff-eb5cfa2a56b5 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:16.474387 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:16.526192 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77033ab-e1fd-4fcb-9cc8-e0b51cd93acf None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:16.529564 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:16.574002 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e98664-8909-41e5-98ff-eb5cfa2a56b5 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:16.586041 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159958b7-9c74-44ea-9a86-881304ef4835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:16.587014 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159958b7-9c74-44ea-9a86-881304ef4835 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:16.587301 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159958b7-9c74-44ea-9a86-881304ef4835 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:16.587575 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159958b7-9c74-44ea-9a86-881304ef4835 None admin] PATH_INFO: `/v3/groups/f26ed4dcd678441abf63c4284a4cc43d` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:16.596095 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159958b7-9c74-44ea-9a86-881304ef4835 None admin] RBAC: Authorizing `identity:get_group(group_id=f26ed4dcd678441abf63c4284a4cc43d)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:16.597756 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159958b7-9c74-44ea-9a86-881304ef4835 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:16.599952 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 192/377] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Jun 2 19:00:16 2022] GET /identity/v3/groups/f26ed4dcd678441abf63c4284a4cc43d => generated 229 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:00:16.614301 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159958b7-9c74-44ea-9a86-881304ef4835 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:16.644347 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a62c3e8-b9a0-4427-90d7-0f8f1a5179fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:16.645223 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a62c3e8-b9a0-4427-90d7-0f8f1a5179fa None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:16.645512 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a62c3e8-b9a0-4427-90d7-0f8f1a5179fa None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:16.645793 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a62c3e8-b9a0-4427-90d7-0f8f1a5179fa None admin] PATH_INFO: `/v3/groups/37365d258edc4d9b852736f631e192fe` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:16.648497 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a62c3e8-b9a0-4427-90d7-0f8f1a5179fa None admin] RBAC: Authorizing `identity:get_group(group_id=37365d258edc4d9b852736f631e192fe)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:16.650054 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a62c3e8-b9a0-4427-90d7-0f8f1a5179fa None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:16.651775 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 186/378] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Jun 2 19:00:16 2022] GET /identity/v3/groups/37365d258edc4d9b852736f631e192fe => generated 226 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:16.660051 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a62c3e8-b9a0-4427-90d7-0f8f1a5179fa None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:16.788174 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159958b7-9c74-44ea-9a86-881304ef4835 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:16.822823 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a62c3e8-b9a0-4427-90d7-0f8f1a5179fa None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:16.841331 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:16.846222 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7452b81-4c24-4565-bfaf-2f53a62d8e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:16.846931 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7452b81-4c24-4565-bfaf-2f53a62d8e62 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:16.847221 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7452b81-4c24-4565-bfaf-2f53a62d8e62 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:16.847573 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7452b81-4c24-4565-bfaf-2f53a62d8e62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:16.848429 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7452b81-4c24-4565-bfaf-2f53a62d8e62 None admin] RBAC: Adding query filter params (group.id=f26ed4dcd678441abf63c4284a4cc43d, role.id=f13fdd0847924014abf1e95594195e78, scope.project.id=879ee62507da4a8282b2418099da56e5) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:16.848950 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7452b81-4c24-4565-bfaf-2f53a62d8e62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:16.850219 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7452b81-4c24-4565-bfaf-2f53a62d8e62 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:16.861157 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 193/379] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Jun 2 19:00:16 2022] GET /identity/v3/role_assignments?group.id=f26ed4dcd678441abf63c4284a4cc43d&scope.project.id=879ee62507da4a8282b2418099da56e5&role.id=f13fdd0847924014abf1e95594195e78 => generated 260 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:16.867549 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbb13da-8cd8-425e-99bb-a2510c641de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:16.868584 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb13da-8cd8-425e-99bb-a2510c641de2 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:16.869074 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb13da-8cd8-425e-99bb-a2510c641de2 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:16.869574 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb13da-8cd8-425e-99bb-a2510c641de2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:16.870629 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbb13da-8cd8-425e-99bb-a2510c641de2 None admin] RBAC: Adding query filter params (group.id=37365d258edc4d9b852736f631e192fe, role.id=2e0c054b646e4ed2a0e7e5962f09a50e, scope.project.id=5f19550a1b254991ba8eb40da4841fcf) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:16.871238 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbb13da-8cd8-425e-99bb-a2510c641de2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:16.873012 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbb13da-8cd8-425e-99bb-a2510c641de2 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:16.880421 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 187/380] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Jun 2 19:00:16 2022] GET /identity/v3/role_assignments?group.id=37365d258edc4d9b852736f631e192fe&scope.project.id=5f19550a1b254991ba8eb40da4841fcf&role.id=2e0c054b646e4ed2a0e7e5962f09a50e => generated 260 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:17.984433 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bb64ff-eb80-4257-9e4c-dd5c72739b5e None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:17.985354 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bb64ff-eb80-4257-9e4c-dd5c72739b5e None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:17.985822 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bb64ff-eb80-4257-9e4c-dd5c72739b5e None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:17.987771 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 194/381] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:00:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:00:18.003462 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9bb628-f19f-4132-b346-c4910cef3a30 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:18.004360 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9bb628-f19f-4132-b346-c4910cef3a30 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:18.004360 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9bb628-f19f-4132-b346-c4910cef3a30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:18.027782 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-6b9bb628-f19f-4132-b346-c4910cef3a30 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:00:18.029017 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-6b9bb628-f19f-4132-b346-c4910cef3a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:00:18.031932 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b34bcf3-d29b-4a7d-ab5f-6dddfdd410c2 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:18.032451 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b34bcf3-d29b-4a7d-ab5f-6dddfdd410c2 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:18.032935 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b34bcf3-d29b-4a7d-ab5f-6dddfdd410c2 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:18.034632 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 195/382] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:00:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:00:18.039929 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f855b8a6-4ce3-4999-bf6c-276164ee8f8d None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:18.040212 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f855b8a6-4ce3-4999-bf6c-276164ee8f8d None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:18.040488 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f855b8a6-4ce3-4999-bf6c-276164ee8f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:18.074635 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-f855b8a6-4ce3-4999-bf6c-276164ee8f8d None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:00:18.076191 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-f855b8a6-4ce3-4999-bf6c-276164ee8f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:00:18.122975 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 188/383] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:00:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 19:00:18.126178 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:18.147334 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9bb628-f19f-4132-b346-c4910cef3a30 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:18.211958 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 196/384] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:00:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 19:00:18.280912 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f855b8a6-4ce3-4999-bf6c-276164ee8f8d None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:18.296319 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9bb628-f19f-4132-b346-c4910cef3a30 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:18.307090 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:18.335217 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44689dee-8215-4317-a307-be9406afa072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:18.336534 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44689dee-8215-4317-a307-be9406afa072 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:18.336778 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44689dee-8215-4317-a307-be9406afa072 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:18.337422 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44689dee-8215-4317-a307-be9406afa072 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:18.349154 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44689dee-8215-4317-a307-be9406afa072 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:00:18.358737 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-44689dee-8215-4317-a307-be9406afa072 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 02 19:00:18.358737 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:18.358737 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:18.358737 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:18.358737 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:18.358737 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:18.358737 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:18.358737 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:18.358737 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:18.358737 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:18.358737 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:18.358737 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:18.358737 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:00:18.358737 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:00:18.358737 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:00:18.358737 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:00:18.359648 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:00:18.359648 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:00:18.359648 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:00:18.359648 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:00:18.359648 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:18.359648 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:18.359648 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:18.359648 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:18.359648 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:18.359648 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:18.359648 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:18.359648 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:18.359648 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:18.359648 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:18.359648 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:18.359648 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:18.360416 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:18.360416 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:18.360416 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:18.360416 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:18.360416 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:00:18.360416 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:00:18.360416 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:00:18.360416 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:00:18.360416 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:00:18.360416 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:00:18.360416 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 02 19:00:18.360416 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:00:18.361223 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 189/385] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 2 19:00:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:18.367421 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44689dee-8215-4317-a307-be9406afa072 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:18.487598 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f855b8a6-4ce3-4999-bf6c-276164ee8f8d None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:18.535221 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ddaa90-72cb-4297-a75c-b669bb3108db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:18.536432 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ddaa90-72cb-4297-a75c-b669bb3108db None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:18.536769 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ddaa90-72cb-4297-a75c-b669bb3108db None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:18.537106 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ddaa90-72cb-4297-a75c-b669bb3108db None admin] PATH_INFO: `/v3/roles/member` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:18.545024 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44689dee-8215-4317-a307-be9406afa072 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:18.549363 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ddaa90-72cb-4297-a75c-b669bb3108db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:00:18.559827 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-59ddaa90-72cb-4297-a75c-b669bb3108db None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 02 19:00:18.559827 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:18.559827 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:18.559827 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:18.559827 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:18.559827 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:18.559827 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:18.559827 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:18.559827 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:18.559827 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:18.559827 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:18.559827 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:18.559827 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:00:18.559827 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:00:18.559827 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:00:18.559827 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:00:18.560736 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:00:18.560736 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:00:18.560736 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:00:18.560736 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:00:18.560736 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:18.560736 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:18.560736 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:18.560736 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:18.560736 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:18.560736 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:18.560736 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:18.560736 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:18.560736 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:18.560736 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:18.560736 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:18.560736 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:18.561723 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:18.561723 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:18.561723 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:18.561723 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:18.561723 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:00:18.561723 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:00:18.561723 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:00:18.561723 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:00:18.561723 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:00:18.561723 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:00:18.561723 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 02 19:00:18.561723 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:00:18.561723 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 197/386] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 2 19:00:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 02 19:00:18.568167 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ddaa90-72cb-4297-a75c-b669bb3108db None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:18.598034 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19b1567-7347-4148-9b38-b72f7e9af491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:18.599107 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b1567-7347-4148-9b38-b72f7e9af491 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:18.599567 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b1567-7347-4148-9b38-b72f7e9af491 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:18.599979 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b1567-7347-4148-9b38-b72f7e9af491 None admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:18.601469 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19b1567-7347-4148-9b38-b72f7e9af491 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:18.601920 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19b1567-7347-4148-9b38-b72f7e9af491 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:18.603481 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19b1567-7347-4148-9b38-b72f7e9af491 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:18.618632 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:18.619227 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 190/387] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 2 19:00:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:18.621002 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19b1567-7347-4148-9b38-b72f7e9af491 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:18.660631 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:18.732896 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ddaa90-72cb-4297-a75c-b669bb3108db None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:18.737071 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19b1567-7347-4148-9b38-b72f7e9af491 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:18.777054 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4008a512-352f-4405-92fe-e16d7be2108c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:18.777675 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c41b92e-94e9-496a-b983-ff0625021526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:18.777823 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4008a512-352f-4405-92fe-e16d7be2108c None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:18.778009 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4008a512-352f-4405-92fe-e16d7be2108c None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:18.778294 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4008a512-352f-4405-92fe-e16d7be2108c None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:18.779079 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4008a512-352f-4405-92fe-e16d7be2108c None admin] RBAC: Adding query filter params (name=member) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:18.779391 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4008a512-352f-4405-92fe-e16d7be2108c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:18.779743 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c41b92e-94e9-496a-b983-ff0625021526 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:18.780230 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c41b92e-94e9-496a-b983-ff0625021526 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:18.780709 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c41b92e-94e9-496a-b983-ff0625021526 None admin] PATH_INFO: `/v3/groups/37365d258edc4d9b852736f631e192fe` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:18.780808 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4008a512-352f-4405-92fe-e16d7be2108c None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:18.782230 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c41b92e-94e9-496a-b983-ff0625021526 None admin] RBAC: Authorizing `identity:get_group(group_id=37365d258edc4d9b852736f631e192fe)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:18.783833 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c41b92e-94e9-496a-b983-ff0625021526 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:18.790807 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 191/389] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Jun 2 19:00:18 2022] GET /identity/v3/groups/37365d258edc4d9b852736f631e192fe => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:18.790991 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 198/389] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 2 19:00:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:18.792402 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0986e2e-d903-40f8-b018-195f1b2a06ed None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:18.792770 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0986e2e-d903-40f8-b018-195f1b2a06ed None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:18.793132 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0986e2e-d903-40f8-b018-195f1b2a06ed None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:18.798849 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 192/390] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:00:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:00:18.800130 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0986e2e-d903-40f8-b018-195f1b2a06ed None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:18.800334 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4008a512-352f-4405-92fe-e16d7be2108c None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:18.888558 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:18.905308 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4008a512-352f-4405-92fe-e16d7be2108c None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:18.910281 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:18.916134 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0986e2e-d903-40f8-b018-195f1b2a06ed None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:18.943972 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5651019-fdb3-42d8-a3b5-a4f7821d919a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:18.945639 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5651019-fdb3-42d8-a3b5-a4f7821d919a None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:18.946115 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5651019-fdb3-42d8-a3b5-a4f7821d919a None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:18.946595 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5651019-fdb3-42d8-a3b5-a4f7821d919a None admin] PATH_INFO: `/v3/projects/abffdc3e7a9447068ceac783b7558c81` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:18.948799 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5651019-fdb3-42d8-a3b5-a4f7821d919a None admin] RBAC: Authorizing `identity:get_project(project_id=abffdc3e7a9447068ceac783b7558c81)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:18.951730 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5651019-fdb3-42d8-a3b5-a4f7821d919a None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:18.954492 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 199/391] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 2 19:00:18 2022] GET /identity/v3/projects/abffdc3e7a9447068ceac783b7558c81 => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:18.960782 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d215c025-c1a3-44f4-9e0e-ef666c960bf6 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:18.961074 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d215c025-c1a3-44f4-9e0e-ef666c960bf6 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:18.961356 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d215c025-c1a3-44f4-9e0e-ef666c960bf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:18.963291 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cf5d8e-05b3-4271-a8ec-fe2a57aa19c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:18.964152 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf5d8e-05b3-4271-a8ec-fe2a57aa19c5 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:18.964439 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf5d8e-05b3-4271-a8ec-fe2a57aa19c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:18.964712 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf5d8e-05b3-4271-a8ec-fe2a57aa19c5 None admin] PATH_INFO: `/v3/groups/37365d258edc4d9b852736f631e192fe` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:18.965905 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cf5d8e-05b3-4271-a8ec-fe2a57aa19c5 None admin] RBAC: Authorizing `identity:get_group(group_id=37365d258edc4d9b852736f631e192fe)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:18.967374 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cf5d8e-05b3-4271-a8ec-fe2a57aa19c5 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:18.969430 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 193/392] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Jun 2 19:00:18 2022] GET /identity/v3/groups/37365d258edc4d9b852736f631e192fe => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:18.969996 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cf5d8e-05b3-4271-a8ec-fe2a57aa19c5 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:18.981000 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-d215c025-c1a3-44f4-9e0e-ef666c960bf6 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:00:18.982547 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-d215c025-c1a3-44f4-9e0e-ef666c960bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:00:19.066558 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 200/393] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:00:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 19:00:19.068426 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d215c025-c1a3-44f4-9e0e-ef666c960bf6 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:19.105977 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cf5d8e-05b3-4271-a8ec-fe2a57aa19c5 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:19.151044 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29b4c43-5966-4940-9fd5-d4e02078b3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:19.152137 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29b4c43-5966-4940-9fd5-d4e02078b3f9 None admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:19.152516 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29b4c43-5966-4940-9fd5-d4e02078b3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:19.152924 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29b4c43-5966-4940-9fd5-d4e02078b3f9 None admin] PATH_INFO: `/v3/projects/abffdc3e7a9447068ceac783b7558c81/groups/37365d258edc4d9b852736f631e192fe/roles/2e0c054b646e4ed2a0e7e5962f09a50e` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:19.156611 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29b4c43-5966-4940-9fd5-d4e02078b3f9 None admin] RBAC: Authorizing `identity:create_grant(project_id=abffdc3e7a9447068ceac783b7558c81, group_id=37365d258edc4d9b852736f631e192fe, role_id=2e0c054b646e4ed2a0e7e5962f09a50e)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:19.158084 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29b4c43-5966-4940-9fd5-d4e02078b3f9 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:19.190320 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:19.208548 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d215c025-c1a3-44f4-9e0e-ef666c960bf6 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:19.222239 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 194/394] 10.222.0.33 () {58 vars in 1442 bytes} [Thu Jun 2 19:00:18 2022] PUT /identity/v3/projects/abffdc3e7a9447068ceac783b7558c81/groups/37365d258edc4d9b852736f631e192fe/roles/2e0c054b646e4ed2a0e7e5962f09a50e => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:00:19.223657 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29b4c43-5966-4940-9fd5-d4e02078b3f9 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:19.282465 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac633a0-871f-4fc3-a9f9-5d529e8b37b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:19.286880 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac633a0-871f-4fc3-a9f9-5d529e8b37b2 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:19.287481 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac633a0-871f-4fc3-a9f9-5d529e8b37b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:19.287992 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac633a0-871f-4fc3-a9f9-5d529e8b37b2 None admin] PATH_INFO: `/v3/projects/abffdc3e7a9447068ceac783b7558c81` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:19.291380 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac633a0-871f-4fc3-a9f9-5d529e8b37b2 None admin] RBAC: Authorizing `identity:get_project(project_id=abffdc3e7a9447068ceac783b7558c81)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:19.293254 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac633a0-871f-4fc3-a9f9-5d529e8b37b2 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:19.300448 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 201/395] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 2 19:00:19 2022] GET /identity/v3/projects/abffdc3e7a9447068ceac783b7558c81 => generated 298 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:19.303466 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac633a0-871f-4fc3-a9f9-5d529e8b37b2 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:19.307251 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:19.462765 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29b4c43-5966-4940-9fd5-d4e02078b3f9 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:19.490400 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac633a0-871f-4fc3-a9f9-5d529e8b37b2 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:19.503194 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e093a3cc-844c-47c5-a90b-7f21f8b70059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:19.503715 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e093a3cc-844c-47c5-a90b-7f21f8b70059 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:19.504004 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e093a3cc-844c-47c5-a90b-7f21f8b70059 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:19.504280 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e093a3cc-844c-47c5-a90b-7f21f8b70059 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:19.515388 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e093a3cc-844c-47c5-a90b-7f21f8b70059 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:00:19.523714 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-e093a3cc-844c-47c5-a90b-7f21f8b70059 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 02 19:00:19.523714 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:19.523714 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:19.523714 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:19.523714 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:19.523714 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:19.523714 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:19.523714 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:19.523714 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:19.523714 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:19.523714 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:19.523714 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:19.523714 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:00:19.523714 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:00:19.523714 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:00:19.523714 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:00:19.524609 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:00:19.524609 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:00:19.524609 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:00:19.524609 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:00:19.524609 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:19.524609 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:19.524609 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:19.524609 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:19.524609 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:19.524609 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:19.524609 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:19.524609 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:19.524609 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:19.524609 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:19.524609 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:19.524609 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:19.525805 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:19.525805 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:19.525805 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:19.525805 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:19.525805 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:00:19.525805 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:00:19.525805 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:00:19.525805 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:00:19.525805 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:00:19.525805 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:00:19.525805 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 02 19:00:19.525805 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:00:19.525805 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 195/396] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 2 19:00:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:19.526493 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cdc3b3-da38-4baa-bd9d-dc555d249961 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:19.526493 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cdc3b3-da38-4baa-bd9d-dc555d249961 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:19.526875 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cdc3b3-da38-4baa-bd9d-dc555d249961 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:19.532310 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 196/397] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:00:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:00:19.533889 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cdc3b3-da38-4baa-bd9d-dc555d249961 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:19.537012 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bc884c-3eb9-49c1-ae88-91227e189f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:19.538880 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bc884c-3eb9-49c1-ae88-91227e189f2d None admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:19.539384 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bc884c-3eb9-49c1-ae88-91227e189f2d None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:19.539857 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bc884c-3eb9-49c1-ae88-91227e189f2d None admin] PATH_INFO: `/v3/projects/abffdc3e7a9447068ceac783b7558c81/groups/37365d258edc4d9b852736f631e192fe/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:19.542151 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bc884c-3eb9-49c1-ae88-91227e189f2d None admin] RBAC: Authorizing `identity:create_grant(project_id=abffdc3e7a9447068ceac783b7558c81, group_id=37365d258edc4d9b852736f631e192fe, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:19.543962 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bc884c-3eb9-49c1-ae88-91227e189f2d None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:19.625068 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:19.641498 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cdc3b3-da38-4baa-bd9d-dc555d249961 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:19.674957 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 202/398] 10.222.0.33 () {58 vars in 1442 bytes} [Thu Jun 2 19:00:19 2022] PUT /identity/v3/projects/abffdc3e7a9447068ceac783b7558c81/groups/37365d258edc4d9b852736f631e192fe/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:00:19.677007 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b96ca1-6e41-498f-8e98-a794a4cfb9f6 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:19.677671 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b96ca1-6e41-498f-8e98-a794a4cfb9f6 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:19.678179 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b96ca1-6e41-498f-8e98-a794a4cfb9f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:19.678772 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0dc3b3-da13-4504-b413-809de9fdd664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:19.679475 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0dc3b3-da13-4504-b413-809de9fdd664 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:19.679760 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0dc3b3-da13-4504-b413-809de9fdd664 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:19.680039 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0dc3b3-da13-4504-b413-809de9fdd664 None admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:19.681023 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0dc3b3-da13-4504-b413-809de9fdd664 None admin] RBAC: Adding query filter params (name=member) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:19.681435 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0dc3b3-da13-4504-b413-809de9fdd664 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:19.682796 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0dc3b3-da13-4504-b413-809de9fdd664 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:19.692449 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 197/399] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 2 19:00:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:00:19.702891 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0dc3b3-da13-4504-b413-809de9fdd664 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:19.721275 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-49b96ca1-6e41-498f-8e98-a794a4cfb9f6 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:00:19.723245 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-49b96ca1-6e41-498f-8e98-a794a4cfb9f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:00:19.731705 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:19.860047 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 203/400] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:00:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 19:00:19.861478 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4fe373-fa53-4793-b806-706c74c5a61a None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:19.861773 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4fe373-fa53-4793-b806-706c74c5a61a None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:19.862052 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4fe373-fa53-4793-b806-706c74c5a61a None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:19.864175 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 204/401] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:00:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:00:19.870590 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0c2f7c-6ed0-4920-836f-ac729439a96b None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:19.871279 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0c2f7c-6ed0-4920-836f-ac729439a96b None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:19.871625 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0c2f7c-6ed0-4920-836f-ac729439a96b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:19.888665 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0dc3b3-da13-4504-b413-809de9fdd664 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:19.896367 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-0d0c2f7c-6ed0-4920-836f-ac729439a96b None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:00:19.898112 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-0d0c2f7c-6ed0-4920-836f-ac729439a96b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:00:19.937816 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1eba47-901d-4a03-91c1-2d8d04a25411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:19.939051 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1eba47-901d-4a03-91c1-2d8d04a25411 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:19.939341 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1eba47-901d-4a03-91c1-2d8d04a25411 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:19.939619 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1eba47-901d-4a03-91c1-2d8d04a25411 None admin] PATH_INFO: `/v3/groups/37365d258edc4d9b852736f631e192fe` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:19.940804 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1eba47-901d-4a03-91c1-2d8d04a25411 None admin] RBAC: Authorizing `identity:get_group(group_id=37365d258edc4d9b852736f631e192fe)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:19.942247 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1eba47-901d-4a03-91c1-2d8d04a25411 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:19.944029 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 198/402] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Jun 2 19:00:19 2022] GET /identity/v3/groups/37365d258edc4d9b852736f631e192fe => generated 226 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:00:19.945794 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1eba47-901d-4a03-91c1-2d8d04a25411 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:19.966461 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:19.993110 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 205/403] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:00:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 19:00:19.994567 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0c2f7c-6ed0-4920-836f-ac729439a96b None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:20.055164 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1eba47-901d-4a03-91c1-2d8d04a25411 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:20.067140 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:20.091660 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2e2e8a-09d9-4d45-8801-ed23d07910f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:20.092623 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2e2e8a-09d9-4d45-8801-ed23d07910f1 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:20.092972 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2e2e8a-09d9-4d45-8801-ed23d07910f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:20.093317 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2e2e8a-09d9-4d45-8801-ed23d07910f1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:20.098163 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0c2f7c-6ed0-4920-836f-ac729439a96b None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:20.105599 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2e2e8a-09d9-4d45-8801-ed23d07910f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:00:20.125064 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-6e2e2e8a-09d9-4d45-8801-ed23d07910f1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 02 19:00:20.125064 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:20.125064 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:20.125064 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:20.125064 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:20.125064 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:20.125064 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:20.125064 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:20.125064 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:20.125064 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:20.125064 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:20.125064 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:20.125064 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:00:20.125064 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:00:20.125064 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:00:20.125064 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:00:20.126439 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:00:20.126439 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:00:20.126439 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:00:20.126439 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:00:20.126439 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:20.126439 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:20.126439 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:20.126439 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:20.126439 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:20.126439 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:20.126439 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:20.126439 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:20.126439 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:20.126439 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:20.126439 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:20.126439 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:20.127260 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:20.127260 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:20.127260 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:20.127260 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:20.127260 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:00:20.127260 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:00:20.127260 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:00:20.127260 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:00:20.127260 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:00:20.127260 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:00:20.127260 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 02 19:00:20.127260 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:00:20.130413 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 199/404] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 2 19:00:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:20.131632 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2e2e8a-09d9-4d45-8801-ed23d07910f1 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:20.140820 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89df50a3-a0cc-4fb3-9a79-1ffd02c2427f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:20.141711 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df50a3-a0cc-4fb3-9a79-1ffd02c2427f None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:20.142002 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df50a3-a0cc-4fb3-9a79-1ffd02c2427f None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:20.142280 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df50a3-a0cc-4fb3-9a79-1ffd02c2427f None admin] PATH_INFO: `/v3/projects/5f19550a1b254991ba8eb40da4841fcf` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:20.143506 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89df50a3-a0cc-4fb3-9a79-1ffd02c2427f None admin] RBAC: Authorizing `identity:get_project(project_id=5f19550a1b254991ba8eb40da4841fcf)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:20.144967 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89df50a3-a0cc-4fb3-9a79-1ffd02c2427f None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:20.146691 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 206/405] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 2 19:00:19 2022] GET /identity/v3/projects/5f19550a1b254991ba8eb40da4841fcf => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:20.148095 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89df50a3-a0cc-4fb3-9a79-1ffd02c2427f None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:20.218500 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:20.252902 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89df50a3-a0cc-4fb3-9a79-1ffd02c2427f None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:20.253898 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2e2e8a-09d9-4d45-8801-ed23d07910f1 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:20.292757 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d93eaea-1163-4626-b055-194830f0d9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:20.293931 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d93eaea-1163-4626-b055-194830f0d9b2 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:20.294510 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d93eaea-1163-4626-b055-194830f0d9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:20.295011 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d93eaea-1163-4626-b055-194830f0d9b2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:20.295241 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed74d83-22a5-43b4-8708-317a3e8112dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:20.296175 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed74d83-22a5-43b4-8708-317a3e8112dc None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:20.296531 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed74d83-22a5-43b4-8708-317a3e8112dc None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:20.296880 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed74d83-22a5-43b4-8708-317a3e8112dc None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:20.297710 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed74d83-22a5-43b4-8708-317a3e8112dc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:20.298270 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed74d83-22a5-43b4-8708-317a3e8112dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:20.300082 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed74d83-22a5-43b4-8708-317a3e8112dc None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:20.306544 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:20.308479 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d93eaea-1163-4626-b055-194830f0d9b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:00:20.311144 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 207/406] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 2 19:00:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:20.312066 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed74d83-22a5-43b4-8708-317a3e8112dc None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:20.317575 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-5d93eaea-1163-4626-b055-194830f0d9b2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 02 19:00:20.317575 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:20.317575 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:20.317575 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:20.317575 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:20.317575 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:20.317575 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:20.317575 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:20.317575 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:20.317575 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:20.317575 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:20.317575 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:20.317575 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:00:20.317575 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:00:20.317575 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:00:20.317575 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:00:20.320682 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:00:20.320682 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:00:20.320682 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:00:20.320682 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:00:20.320682 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:20.320682 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:20.320682 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:20.320682 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:20.320682 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:20.320682 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:20.320682 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:20.320682 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:20.320682 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:20.320682 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:20.320682 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:20.320682 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:20.321474 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:20.321474 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:20.321474 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:20.321474 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:20.321474 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:00:20.321474 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:00:20.321474 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:00:20.321474 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:00:20.321474 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:00:20.321474 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:00:20.321474 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 02 19:00:20.321474 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:00:20.321474 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 200/407] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 2 19:00:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:20.322967 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d93eaea-1163-4626-b055-194830f0d9b2 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:20.434081 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed74d83-22a5-43b4-8708-317a3e8112dc None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:20.442990 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d93eaea-1163-4626-b055-194830f0d9b2 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:20.474084 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222a7a75-fbbe-40e7-a343-2099454a06c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:20.475017 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222a7a75-fbbe-40e7-a343-2099454a06c1 None admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:20.475303 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222a7a75-fbbe-40e7-a343-2099454a06c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:20.476275 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222a7a75-fbbe-40e7-a343-2099454a06c1 None admin] PATH_INFO: `/v3/projects/5f19550a1b254991ba8eb40da4841fcf/groups/37365d258edc4d9b852736f631e192fe/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:20.478431 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222a7a75-fbbe-40e7-a343-2099454a06c1 None admin] RBAC: Authorizing `identity:create_grant(project_id=5f19550a1b254991ba8eb40da4841fcf, group_id=37365d258edc4d9b852736f631e192fe, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:20.480015 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222a7a75-fbbe-40e7-a343-2099454a06c1 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:20.482742 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4ab906-2c1f-4c22-886b-d3a20e7b1838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:20.483878 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ab906-2c1f-4c22-886b-d3a20e7b1838 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:20.484365 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ab906-2c1f-4c22-886b-d3a20e7b1838 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:20.484858 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ab906-2c1f-4c22-886b-d3a20e7b1838 None admin] PATH_INFO: `/v3/groups/37365d258edc4d9b852736f631e192fe` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:20.486145 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4ab906-2c1f-4c22-886b-d3a20e7b1838 None admin] RBAC: Authorizing `identity:get_group(group_id=37365d258edc4d9b852736f631e192fe)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:20.487706 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4ab906-2c1f-4c22-886b-d3a20e7b1838 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:20.489590 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 201/408] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Jun 2 19:00:20 2022] GET /identity/v3/groups/37365d258edc4d9b852736f631e192fe => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:20.490403 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4ab906-2c1f-4c22-886b-d3a20e7b1838 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:20.494780 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:20.586166 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 208/409] 10.222.0.33 () {58 vars in 1442 bytes} [Thu Jun 2 19:00:20 2022] PUT /identity/v3/projects/5f19550a1b254991ba8eb40da4841fcf/groups/37365d258edc4d9b852736f631e192fe/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:00:20.587555 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222a7a75-fbbe-40e7-a343-2099454a06c1 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:20.608073 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4ab906-2c1f-4c22-886b-d3a20e7b1838 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:20.670464 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:20.692122 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d25d1e8-c63d-4d00-a65c-11487f54ede0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:20.692821 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25d1e8-c63d-4d00-a65c-11487f54ede0 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:20.693113 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25d1e8-c63d-4d00-a65c-11487f54ede0 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:20.693397 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25d1e8-c63d-4d00-a65c-11487f54ede0 None admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:20.694706 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d25d1e8-c63d-4d00-a65c-11487f54ede0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:20.695051 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d25d1e8-c63d-4d00-a65c-11487f54ede0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:20.696446 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d25d1e8-c63d-4d00-a65c-11487f54ede0 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:20.717581 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 202/410] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 2 19:00:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:20.723620 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d25d1e8-c63d-4d00-a65c-11487f54ede0 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:20.745634 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222a7a75-fbbe-40e7-a343-2099454a06c1 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:20.832444 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:20.845741 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb12db42-66dc-4118-86cc-638c852bbad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:20.847026 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb12db42-66dc-4118-86cc-638c852bbad5 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:20.847524 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb12db42-66dc-4118-86cc-638c852bbad5 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:20.848076 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb12db42-66dc-4118-86cc-638c852bbad5 None admin] PATH_INFO: `/v3/projects/5f19550a1b254991ba8eb40da4841fcf` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:20.849477 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb12db42-66dc-4118-86cc-638c852bbad5 None admin] RBAC: Authorizing `identity:get_project(project_id=5f19550a1b254991ba8eb40da4841fcf)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:20.851297 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb12db42-66dc-4118-86cc-638c852bbad5 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:20.853534 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 209/411] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 2 19:00:20 2022] GET /identity/v3/projects/5f19550a1b254991ba8eb40da4841fcf => generated 302 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:20.859319 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb12db42-66dc-4118-86cc-638c852bbad5 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:20.873217 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d25d1e8-c63d-4d00-a65c-11487f54ede0 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:20.931017 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0d6bae-8f05-42e1-b616-ef58bcfb75a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:20.932824 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0d6bae-8f05-42e1-b616-ef58bcfb75a7 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:20.933532 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0d6bae-8f05-42e1-b616-ef58bcfb75a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:20.934017 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0d6bae-8f05-42e1-b616-ef58bcfb75a7 None admin] PATH_INFO: `/v3/groups/f26ed4dcd678441abf63c4284a4cc43d` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:20.936120 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0d6bae-8f05-42e1-b616-ef58bcfb75a7 None admin] RBAC: Authorizing `identity:get_group(group_id=f26ed4dcd678441abf63c4284a4cc43d)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:20.938276 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0d6bae-8f05-42e1-b616-ef58bcfb75a7 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:20.940878 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 203/412] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Jun 2 19:00:20 2022] GET /identity/v3/groups/f26ed4dcd678441abf63c4284a4cc43d => generated 229 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:20.946565 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0d6bae-8f05-42e1-b616-ef58bcfb75a7 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:21.001649 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb12db42-66dc-4118-86cc-638c852bbad5 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:21.039688 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:21.047752 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3154ac-fae7-4318-b72c-92ce6192b6b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:21.049037 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3154ac-fae7-4318-b72c-92ce6192b6b2 None admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:21.049148 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3154ac-fae7-4318-b72c-92ce6192b6b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:21.049503 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3154ac-fae7-4318-b72c-92ce6192b6b2 None admin] PATH_INFO: `/v3/projects/5f19550a1b254991ba8eb40da4841fcf/groups/37365d258edc4d9b852736f631e192fe/roles/2e0c054b646e4ed2a0e7e5962f09a50e` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:21.057640 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3154ac-fae7-4318-b72c-92ce6192b6b2 None admin] RBAC: Authorizing `identity:create_grant(project_id=5f19550a1b254991ba8eb40da4841fcf, group_id=37365d258edc4d9b852736f631e192fe, role_id=2e0c054b646e4ed2a0e7e5962f09a50e)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:21.057640 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3154ac-fae7-4318-b72c-92ce6192b6b2 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:21.078639 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0d6bae-8f05-42e1-b616-ef58bcfb75a7 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:21.127223 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9986596-8ef8-4e7b-92bc-980ce6021f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:21.129099 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9986596-8ef8-4e7b-92bc-980ce6021f8e None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:21.129535 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9986596-8ef8-4e7b-92bc-980ce6021f8e None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:21.129962 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9986596-8ef8-4e7b-92bc-980ce6021f8e None admin] PATH_INFO: `/v3/projects/879ee62507da4a8282b2418099da56e5` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:21.131511 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9986596-8ef8-4e7b-92bc-980ce6021f8e None admin] RBAC: Authorizing `identity:get_project(project_id=879ee62507da4a8282b2418099da56e5)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:21.135356 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9986596-8ef8-4e7b-92bc-980ce6021f8e None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:21.136125 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 204/413] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 2 19:00:20 2022] GET /identity/v3/projects/879ee62507da4a8282b2418099da56e5 => generated 344 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:00:21.142822 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9986596-8ef8-4e7b-92bc-980ce6021f8e None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:21.147064 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:21.165513 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 210/414] 10.222.0.33 () {58 vars in 1442 bytes} [Thu Jun 2 19:00:20 2022] PUT /identity/v3/projects/5f19550a1b254991ba8eb40da4841fcf/groups/37365d258edc4d9b852736f631e192fe/roles/2e0c054b646e4ed2a0e7e5962f09a50e => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 02 19:00:21.270791 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9986596-8ef8-4e7b-92bc-980ce6021f8e None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:21.311074 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c75e37c-3edc-41f4-9e33-26bc00fcbf16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:21.312056 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c75e37c-3edc-41f4-9e33-26bc00fcbf16 None admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:21.312349 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c75e37c-3edc-41f4-9e33-26bc00fcbf16 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:21.312632 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c75e37c-3edc-41f4-9e33-26bc00fcbf16 None admin] PATH_INFO: `/v3/projects/879ee62507da4a8282b2418099da56e5/groups/f26ed4dcd678441abf63c4284a4cc43d/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:21.314851 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c75e37c-3edc-41f4-9e33-26bc00fcbf16 None admin] RBAC: Authorizing `identity:create_grant(project_id=879ee62507da4a8282b2418099da56e5, group_id=f26ed4dcd678441abf63c4284a4cc43d, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:21.316347 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c75e37c-3edc-41f4-9e33-26bc00fcbf16 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:21.414237 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 205/415] 10.222.0.33 () {58 vars in 1442 bytes} [Thu Jun 2 19:00:21 2022] PUT /identity/v3/projects/879ee62507da4a8282b2418099da56e5/groups/f26ed4dcd678441abf63c4284a4cc43d/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:00:21.714328 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d198cad-300f-4e07-b437-cec30a7dba94 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:21.715033 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d198cad-300f-4e07-b437-cec30a7dba94 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:21.715534 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d198cad-300f-4e07-b437-cec30a7dba94 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:21.717169 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 211/416] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:00:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:00:21.723272 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902036b3-e1b9-442d-bf6f-cf14b771964f None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:21.723574 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902036b3-e1b9-442d-bf6f-cf14b771964f None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:21.723856 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902036b3-e1b9-442d-bf6f-cf14b771964f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:21.747490 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-902036b3-e1b9-442d-bf6f-cf14b771964f None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:00:21.749348 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-902036b3-e1b9-442d-bf6f-cf14b771964f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:00:21.830677 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:21.845939 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 206/417] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:00:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 19:00:21.892296 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d198cad-300f-4e07-b437-cec30a7dba94 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:22.019489 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d198cad-300f-4e07-b437-cec30a7dba94 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:22.059866 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e533a95-9170-4381-abaf-9697d92baf63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:22.060975 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e533a95-9170-4381-abaf-9697d92baf63 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:22.061440 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e533a95-9170-4381-abaf-9697d92baf63 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:22.061894 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e533a95-9170-4381-abaf-9697d92baf63 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:22.077220 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e533a95-9170-4381-abaf-9697d92baf63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:00:22.090221 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-8e533a95-9170-4381-abaf-9697d92baf63 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 02 19:00:22.090221 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:22.090221 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:22.090221 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:22.090221 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:22.090221 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:22.090221 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:22.090221 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:22.090221 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:22.090221 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:22.090221 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:22.090221 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:22.090221 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:00:22.090221 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:00:22.090221 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:00:22.090221 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:00:22.091208 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:00:22.091208 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:00:22.091208 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:00:22.091208 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:00:22.091208 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:22.091208 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:22.091208 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:22.091208 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:22.091208 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:22.091208 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:22.091208 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:22.091208 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:22.091208 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:22.091208 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:22.091208 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:22.091208 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:22.092076 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:22.092076 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:22.092076 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:22.092076 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:22.092076 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:00:22.092076 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:00:22.092076 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:00:22.092076 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:00:22.092076 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:00:22.092076 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:00:22.092076 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 02 19:00:22.092076 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:00:22.094268 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 212/418] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 2 19:00:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:22.101180 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902036b3-e1b9-442d-bf6f-cf14b771964f None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:22.229461 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326bca75-d120-475a-8b22-b376159c7f70 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:22.230503 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326bca75-d120-475a-8b22-b376159c7f70 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:22.231053 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326bca75-d120-475a-8b22-b376159c7f70 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:22.245124 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902036b3-e1b9-442d-bf6f-cf14b771964f None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:22.248131 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 213/419] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:00:22 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:00:22.253539 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52be3a38-406e-4eb3-9a3f-049298cff087 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:22.254124 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52be3a38-406e-4eb3-9a3f-049298cff087 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:22.254595 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52be3a38-406e-4eb3-9a3f-049298cff087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:22.280960 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-52be3a38-406e-4eb3-9a3f-049298cff087 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:00:22.282088 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-52be3a38-406e-4eb3-9a3f-049298cff087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:00:22.289225 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc6667c-80d6-4a2a-9339-26ab60d33790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:22.289965 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc6667c-80d6-4a2a-9339-26ab60d33790 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:22.290251 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc6667c-80d6-4a2a-9339-26ab60d33790 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:22.290523 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc6667c-80d6-4a2a-9339-26ab60d33790 None admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:22.291314 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc6667c-80d6-4a2a-9339-26ab60d33790 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:22.291620 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc6667c-80d6-4a2a-9339-26ab60d33790 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:22.293075 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc6667c-80d6-4a2a-9339-26ab60d33790 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:22.313274 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 207/420] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 2 19:00:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:22.320500 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc6667c-80d6-4a2a-9339-26ab60d33790 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:22.392587 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:22.395143 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:22.419821 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 214/421] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:00:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 02 19:00:22.428815 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc6667c-80d6-4a2a-9339-26ab60d33790 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:22.460624 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52be3a38-406e-4eb3-9a3f-049298cff087 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:22.469392 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cd1b66-382c-4ba1-a08a-aafd6750abde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:22.470052 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cd1b66-382c-4ba1-a08a-aafd6750abde None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:22.470344 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cd1b66-382c-4ba1-a08a-aafd6750abde None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:22.470633 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cd1b66-382c-4ba1-a08a-aafd6750abde None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:22.608465 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cd1b66-382c-4ba1-a08a-aafd6750abde None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:22.609924 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cd1b66-382c-4ba1-a08a-aafd6750abde None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:22.675109 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:22.718912 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 208/422] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 2 19:00:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 02 19:00:22.724938 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cd1b66-382c-4ba1-a08a-aafd6750abde None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:22.730364 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52be3a38-406e-4eb3-9a3f-049298cff087 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:22.822904 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0ffb66-38d0-4383-b7d4-c3894f8be958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:22.823738 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0ffb66-38d0-4383-b7d4-c3894f8be958 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:22.824026 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0ffb66-38d0-4383-b7d4-c3894f8be958 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:22.824303 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0ffb66-38d0-4383-b7d4-c3894f8be958 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:22.859571 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cd1b66-382c-4ba1-a08a-aafd6750abde None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:22.863429 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0ffb66-38d0-4383-b7d4-c3894f8be958 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:00:22.871306 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-cc0ffb66-38d0-4383-b7d4-c3894f8be958 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 02 19:00:22.871306 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:22.871306 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:22.871306 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:22.871306 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:22.871306 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:22.871306 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:22.871306 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:22.871306 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:22.871306 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:22.871306 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:22.871306 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:22.871306 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:00:22.871306 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:00:22.871306 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:00:22.871306 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:00:22.872211 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:00:22.872211 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:00:22.872211 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:00:22.872211 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:00:22.872211 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:22.872211 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:22.872211 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:22.872211 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:22.872211 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:22.872211 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:22.872211 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:22.872211 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:22.872211 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:22.872211 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:22.872211 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:22.872211 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:22.873193 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:22.873193 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:22.873193 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:22.873193 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:22.873193 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:00:22.873193 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:00:22.873193 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:00:22.873193 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:00:22.873193 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:00:22.873193 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:00:22.873193 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 02 19:00:22.873193 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:00:22.873193 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 215/423] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 2 19:00:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 413 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:22.883742 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0ffb66-38d0-4383-b7d4-c3894f8be958 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:22.903722 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402ecaea-6e36-490b-a1ea-69dc9e560204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:22.904602 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402ecaea-6e36-490b-a1ea-69dc9e560204 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:22.904892 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402ecaea-6e36-490b-a1ea-69dc9e560204 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:22.905169 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402ecaea-6e36-490b-a1ea-69dc9e560204 None admin] PATH_INFO: `/v3/projects/abffdc3e7a9447068ceac783b7558c81` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:22.908105 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402ecaea-6e36-490b-a1ea-69dc9e560204 None admin] RBAC: Authorizing `identity:get_project(project_id=abffdc3e7a9447068ceac783b7558c81)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:22.909560 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402ecaea-6e36-490b-a1ea-69dc9e560204 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:22.911638 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 209/424] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 2 19:00:22 2022] GET /identity/v3/projects/abffdc3e7a9447068ceac783b7558c81 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:22.917093 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402ecaea-6e36-490b-a1ea-69dc9e560204 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:22.963102 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:22.971132 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:23.088245 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0ffb66-38d0-4383-b7d4-c3894f8be958 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:23.113944 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402ecaea-6e36-490b-a1ea-69dc9e560204 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:23.149534 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994f60cd-e89c-4115-858e-a9ce9d79b13e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:23.150402 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f60cd-e89c-4115-858e-a9ce9d79b13e None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:23.150692 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f60cd-e89c-4115-858e-a9ce9d79b13e None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:23.151015 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f60cd-e89c-4115-858e-a9ce9d79b13e None admin] PATH_INFO: `/v3/groups/37365d258edc4d9b852736f631e192fe` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:23.152214 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994f60cd-e89c-4115-858e-a9ce9d79b13e None admin] RBAC: Authorizing `identity:get_group(group_id=37365d258edc4d9b852736f631e192fe)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:23.152314 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc0145e-89b7-469b-8cf3-67f09d2502c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:23.152918 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc0145e-89b7-469b-8cf3-67f09d2502c1 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:23.153212 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc0145e-89b7-469b-8cf3-67f09d2502c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:23.153497 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc0145e-89b7-469b-8cf3-67f09d2502c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:23.153671 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994f60cd-e89c-4115-858e-a9ce9d79b13e None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:23.154340 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc0145e-89b7-469b-8cf3-67f09d2502c1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:23.154656 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc0145e-89b7-469b-8cf3-67f09d2502c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:23.155754 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 210/425] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Jun 2 19:00:22 2022] GET /identity/v3/groups/37365d258edc4d9b852736f631e192fe => generated 226 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:00:23.156063 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc0145e-89b7-469b-8cf3-67f09d2502c1 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:23.161719 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994f60cd-e89c-4115-858e-a9ce9d79b13e None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:23.173301 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 216/426] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 2 19:00:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:23.178374 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc0145e-89b7-469b-8cf3-67f09d2502c1 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:23.265860 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994f60cd-e89c-4115-858e-a9ce9d79b13e None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:23.291116 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:23.305004 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96da88ef-4809-4d68-ad8b-5dea7f55c7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:23.306276 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da88ef-4809-4d68-ad8b-5dea7f55c7d7 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:23.313725 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da88ef-4809-4d68-ad8b-5dea7f55c7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:23.313725 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da88ef-4809-4d68-ad8b-5dea7f55c7d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:23.313725 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96da88ef-4809-4d68-ad8b-5dea7f55c7d7 None admin] RBAC: Adding query filter params (group.id=37365d258edc4d9b852736f631e192fe, role.id=2e0c054b646e4ed2a0e7e5962f09a50e, scope.project.id=abffdc3e7a9447068ceac783b7558c81) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:23.313725 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96da88ef-4809-4d68-ad8b-5dea7f55c7d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:23.313725 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96da88ef-4809-4d68-ad8b-5dea7f55c7d7 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:23.317808 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 211/427] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Jun 2 19:00:23 2022] GET /identity/v3/role_assignments?group.id=37365d258edc4d9b852736f631e192fe&scope.project.id=abffdc3e7a9447068ceac783b7558c81&role.id=2e0c054b646e4ed2a0e7e5962f09a50e => generated 611 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:00:23.398598 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:23.402595 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc0145e-89b7-469b-8cf3-67f09d2502c1 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:23.458743 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4afb10e-3dd3-4333-b9de-601c3b1f738e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:23.459488 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4afb10e-3dd3-4333-b9de-601c3b1f738e None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:23.459778 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4afb10e-3dd3-4333-b9de-601c3b1f738e None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:23.460094 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4afb10e-3dd3-4333-b9de-601c3b1f738e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:23.566474 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4afb10e-3dd3-4333-b9de-601c3b1f738e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:23.568051 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4afb10e-3dd3-4333-b9de-601c3b1f738e None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:23.653157 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d32a5ff-fc75-4037-b9f7-2866091f94a2 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:23.653713 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d32a5ff-fc75-4037-b9f7-2866091f94a2 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:23.654193 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d32a5ff-fc75-4037-b9f7-2866091f94a2 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:23.655856 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 212/428] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:00:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 02 19:00:23.664766 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c624870-d24e-4cf3-9f15-6bcbd143a4e7 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:23.665492 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c624870-d24e-4cf3-9f15-6bcbd143a4e7 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:23.665994 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c624870-d24e-4cf3-9f15-6bcbd143a4e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:23.689663 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-3c624870-d24e-4cf3-9f15-6bcbd143a4e7 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:00:23.691593 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-3c624870-d24e-4cf3-9f15-6bcbd143a4e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:00:23.700225 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:23.704421 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 217/429] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 2 19:00:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 02 19:00:23.709502 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4afb10e-3dd3-4333-b9de-601c3b1f738e None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:23.787748 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 213/430] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:00:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 19:00:23.815424 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c624870-d24e-4cf3-9f15-6bcbd143a4e7 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:23.856210 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4afb10e-3dd3-4333-b9de-601c3b1f738e None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:23.894336 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6cfd7d-467c-495f-b65a-fa34e6aa9d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:23.895516 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6cfd7d-467c-495f-b65a-fa34e6aa9d42 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:23.895699 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6cfd7d-467c-495f-b65a-fa34e6aa9d42 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:23.895921 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6cfd7d-467c-495f-b65a-fa34e6aa9d42 None admin] PATH_INFO: `/v3/projects/abffdc3e7a9447068ceac783b7558c81` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:23.897227 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6cfd7d-467c-495f-b65a-fa34e6aa9d42 None admin] RBAC: Authorizing `identity:get_project(project_id=abffdc3e7a9447068ceac783b7558c81)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:23.898705 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6cfd7d-467c-495f-b65a-fa34e6aa9d42 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:23.901083 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 218/431] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 2 19:00:23 2022] GET /identity/v3/projects/abffdc3e7a9447068ceac783b7558c81 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:00:23.905919 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6cfd7d-467c-495f-b65a-fa34e6aa9d42 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:23.920734 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:23.938871 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c624870-d24e-4cf3-9f15-6bcbd143a4e7 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:23.976621 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627a7108-53d1-438a-82ee-0bf16fd18d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:23.977799 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627a7108-53d1-438a-82ee-0bf16fd18d19 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:23.978295 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627a7108-53d1-438a-82ee-0bf16fd18d19 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:23.978782 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627a7108-53d1-438a-82ee-0bf16fd18d19 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:23.980229 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:23.990563 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627a7108-53d1-438a-82ee-0bf16fd18d19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:00:23.998882 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-627a7108-53d1-438a-82ee-0bf16fd18d19 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 02 19:00:23.998882 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:23.998882 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:23.998882 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:23.998882 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:23.998882 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:23.998882 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:23.998882 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:23.998882 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:23.998882 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:23.998882 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:23.998882 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:23.998882 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:00:23.998882 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:00:23.998882 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:00:23.998882 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:00:23.999837 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:00:23.999837 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:00:23.999837 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:00:23.999837 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:00:23.999837 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:23.999837 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:23.999837 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:23.999837 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:23.999837 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:23.999837 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:23.999837 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:23.999837 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:23.999837 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:23.999837 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:23.999837 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:23.999837 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:24.000649 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:24.000649 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:24.000649 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:24.000649 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:24.000649 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:00:24.000649 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:00:24.000649 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:00:24.000649 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:00:24.000649 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:00:24.000649 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:00:24.000649 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 02 19:00:24.000649 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:00:24.000649 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 214/432] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 2 19:00:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 02 19:00:24.001627 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f8776e-2df3-48d0-8b39-085c6009df0a None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:24.001627 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f8776e-2df3-48d0-8b39-085c6009df0a None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:24.001876 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f8776e-2df3-48d0-8b39-085c6009df0a None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:24.005888 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 215/433] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:00:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:00:24.006135 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f8776e-2df3-48d0-8b39-085c6009df0a None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:24.027225 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6cfd7d-467c-495f-b65a-fa34e6aa9d42 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:24.063960 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3f802c-9deb-4e89-9c12-c6494c00674f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:24.064988 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3f802c-9deb-4e89-9c12-c6494c00674f None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:24.065275 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3f802c-9deb-4e89-9c12-c6494c00674f None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:24.065555 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3f802c-9deb-4e89-9c12-c6494c00674f None admin] PATH_INFO: `/v3/groups/37365d258edc4d9b852736f631e192fe` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:24.066874 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3f802c-9deb-4e89-9c12-c6494c00674f None admin] RBAC: Authorizing `identity:get_group(group_id=37365d258edc4d9b852736f631e192fe)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:24.068620 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3f802c-9deb-4e89-9c12-c6494c00674f None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:24.075198 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 219/434] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Jun 2 19:00:23 2022] GET /identity/v3/groups/37365d258edc4d9b852736f631e192fe => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:24.075198 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfb5239-97cc-4e54-96d1-13c571bc945a None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:24.075198 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfb5239-97cc-4e54-96d1-13c571bc945a None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:24.075198 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfb5239-97cc-4e54-96d1-13c571bc945a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:24.090478 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-3cfb5239-97cc-4e54-96d1-13c571bc945a None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:00:24.091761 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-3cfb5239-97cc-4e54-96d1-13c571bc945a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:00:24.124541 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f8776e-2df3-48d0-8b39-085c6009df0a None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:24.159715 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707b0ce0-3a1a-4aff-9524-50cd6d83481b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:24.160301 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b0ce0-3a1a-4aff-9524-50cd6d83481b None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:24.160615 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b0ce0-3a1a-4aff-9524-50cd6d83481b None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:24.160886 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b0ce0-3a1a-4aff-9524-50cd6d83481b None admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:24.161632 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707b0ce0-3a1a-4aff-9524-50cd6d83481b None admin] RBAC: Adding query filter params (name=member) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:24.161933 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707b0ce0-3a1a-4aff-9524-50cd6d83481b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:24.163978 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707b0ce0-3a1a-4aff-9524-50cd6d83481b None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:24.166909 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 220/435] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:00:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 19:00:24.167752 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfb5239-97cc-4e54-96d1-13c571bc945a None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:24.173448 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 216/436] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 2 19:00:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:00:24.174723 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5a2ec4-aeae-4283-9eee-e8dce9b42342 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:24.175037 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5a2ec4-aeae-4283-9eee-e8dce9b42342 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:24.175314 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5a2ec4-aeae-4283-9eee-e8dce9b42342 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:24.176683 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 217/437] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:00:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:00:24.179767 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5a2ec4-aeae-4283-9eee-e8dce9b42342 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:24.189980 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:24.236139 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:24.270953 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfb5239-97cc-4e54-96d1-13c571bc945a None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:24.285528 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5a2ec4-aeae-4283-9eee-e8dce9b42342 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:24.304918 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5329ea1-1cad-40e7-9728-18c311582c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:24.305576 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5329ea1-1cad-40e7-9728-18c311582c64 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:24.305854 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5329ea1-1cad-40e7-9728-18c311582c64 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:24.306140 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5329ea1-1cad-40e7-9728-18c311582c64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:24.307012 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5329ea1-1cad-40e7-9728-18c311582c64 None admin] RBAC: Adding query filter params (group.id=37365d258edc4d9b852736f631e192fe, role.id=77d5301ed78146468d4115b512cc70b7, scope.project.id=abffdc3e7a9447068ceac783b7558c81) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:24.307331 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5329ea1-1cad-40e7-9728-18c311582c64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:24.308780 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5329ea1-1cad-40e7-9728-18c311582c64 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:24.314472 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 221/438] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Jun 2 19:00:24 2022] GET /identity/v3/role_assignments?group.id=37365d258edc4d9b852736f631e192fe&scope.project.id=abffdc3e7a9447068ceac783b7558c81&role.id=77d5301ed78146468d4115b512cc70b7 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:24.315690 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9af834-dcee-4a2e-bc34-d4b54feaea4e None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:24.315981 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9af834-dcee-4a2e-bc34-d4b54feaea4e None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:24.316261 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9af834-dcee-4a2e-bc34-d4b54feaea4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:24.329159 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47426c3-7aa9-498a-8269-4b19b1df8aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:24.329897 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47426c3-7aa9-498a-8269-4b19b1df8aad None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:24.330183 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47426c3-7aa9-498a-8269-4b19b1df8aad None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:24.330701 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47426c3-7aa9-498a-8269-4b19b1df8aad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:24.334354 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-4c9af834-dcee-4a2e-bc34-d4b54feaea4e None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:00:24.335719 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-4c9af834-dcee-4a2e-bc34-d4b54feaea4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:00:24.420072 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47426c3-7aa9-498a-8269-4b19b1df8aad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:24.421594 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47426c3-7aa9-498a-8269-4b19b1df8aad None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:24.434219 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 222/439] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:00:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 19:00:24.436168 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9af834-dcee-4a2e-bc34-d4b54feaea4e None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:24.447159 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:24.512240 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:24.544598 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 218/440] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 2 19:00:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 02 19:00:24.545816 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47426c3-7aa9-498a-8269-4b19b1df8aad None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:24.567612 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9af834-dcee-4a2e-bc34-d4b54feaea4e None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:24.612115 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d6687c-86d2-4c29-9f23-b5dd4636f15a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:24.612929 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d6687c-86d2-4c29-9f23-b5dd4636f15a None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:24.613210 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d6687c-86d2-4c29-9f23-b5dd4636f15a None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:24.613483 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d6687c-86d2-4c29-9f23-b5dd4636f15a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:24.627626 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d6687c-86d2-4c29-9f23-b5dd4636f15a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:00:24.636776 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-74d6687c-86d2-4c29-9f23-b5dd4636f15a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 02 19:00:24.636776 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:24.636776 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:24.636776 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:24.636776 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:24.636776 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:24.636776 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:24.636776 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:24.636776 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:24.636776 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:24.636776 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:24.636776 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:24.636776 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:00:24.636776 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:00:24.636776 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:00:24.636776 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:00:24.637784 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:00:24.637784 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:00:24.637784 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:00:24.637784 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:00:24.637784 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:24.637784 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:24.637784 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:24.637784 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:24.637784 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:24.637784 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:24.637784 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:24.637784 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:24.637784 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:24.637784 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:24.637784 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:24.637784 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:24.638664 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:24.638664 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:24.638664 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:24.638664 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:24.638664 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:00:24.638664 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:00:24.638664 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:00:24.638664 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:00:24.638664 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:00:24.638664 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:00:24.638664 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 02 19:00:24.638664 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:00:24.638664 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 223/441] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 2 19:00:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:24.639680 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d6687c-86d2-4c29-9f23-b5dd4636f15a None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:24.685241 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47426c3-7aa9-498a-8269-4b19b1df8aad None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:24.695140 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:24.725324 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81d657c-5ff3-46e8-951d-83ffc663caa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:24.726719 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81d657c-5ff3-46e8-951d-83ffc663caa4 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:24.727120 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81d657c-5ff3-46e8-951d-83ffc663caa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:24.727393 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81d657c-5ff3-46e8-951d-83ffc663caa4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:24.741143 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81d657c-5ff3-46e8-951d-83ffc663caa4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:00:24.749983 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-a81d657c-5ff3-46e8-951d-83ffc663caa4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 02 19:00:24.749983 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:24.749983 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:24.749983 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:24.749983 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:24.749983 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:24.749983 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:24.749983 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:24.749983 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:24.749983 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:24.749983 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:24.749983 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:24.749983 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:00:24.749983 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:00:24.749983 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:00:24.749983 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:00:24.751214 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:00:24.751214 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:00:24.751214 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:00:24.751214 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:00:24.751214 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:24.751214 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:24.751214 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:24.751214 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:24.751214 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:24.751214 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:24.751214 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:24.751214 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:24.751214 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:24.751214 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:24.751214 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:24.751214 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:24.752180 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:24.752180 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:24.752180 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:24.752180 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:24.752180 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:00:24.752180 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:00:24.752180 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:00:24.752180 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:00:24.752180 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:00:24.752180 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:00:24.752180 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 02 19:00:24.752180 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:00:24.752180 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 219/442] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 2 19:00:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:24.752180 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81d657c-5ff3-46e8-951d-83ffc663caa4 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:24.763208 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d6687c-86d2-4c29-9f23-b5dd4636f15a None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:24.787263 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:24.807061 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5d5660-4a30-4e68-a66f-b68f9e6220cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:24.808208 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5d5660-4a30-4e68-a66f-b68f9e6220cf None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:24.808611 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5d5660-4a30-4e68-a66f-b68f9e6220cf None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:24.808903 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5d5660-4a30-4e68-a66f-b68f9e6220cf None admin] PATH_INFO: `/v3/projects/5f19550a1b254991ba8eb40da4841fcf` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:24.810125 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5d5660-4a30-4e68-a66f-b68f9e6220cf None admin] RBAC: Authorizing `identity:get_project(project_id=5f19550a1b254991ba8eb40da4841fcf)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:24.811614 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5d5660-4a30-4e68-a66f-b68f9e6220cf None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:24.814148 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 224/443] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 2 19:00:24 2022] GET /identity/v3/projects/5f19550a1b254991ba8eb40da4841fcf => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:24.814296 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5d5660-4a30-4e68-a66f-b68f9e6220cf None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:24.883020 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81d657c-5ff3-46e8-951d-83ffc663caa4 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:24.922286 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3269f0-4e76-4319-8dc8-b3bd781dafb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:24.923226 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3269f0-4e76-4319-8dc8-b3bd781dafb5 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:24.923509 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3269f0-4e76-4319-8dc8-b3bd781dafb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:24.923786 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3269f0-4e76-4319-8dc8-b3bd781dafb5 None admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:24.924529 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3269f0-4e76-4319-8dc8-b3bd781dafb5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:24.924837 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3269f0-4e76-4319-8dc8-b3bd781dafb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:24.926197 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3269f0-4e76-4319-8dc8-b3bd781dafb5 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:24.926968 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5d5660-4a30-4e68-a66f-b68f9e6220cf None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:24.934124 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 220/444] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 2 19:00:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:24.934939 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3269f0-4e76-4319-8dc8-b3bd781dafb5 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:24.962449 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:24.965912 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee5ff71-21ac-4c7a-9d25-c152a4022507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:24.966588 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5ff71-21ac-4c7a-9d25-c152a4022507 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:24.966905 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5ff71-21ac-4c7a-9d25-c152a4022507 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:24.967178 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5ff71-21ac-4c7a-9d25-c152a4022507 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:24.967924 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee5ff71-21ac-4c7a-9d25-c152a4022507 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:24.968229 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee5ff71-21ac-4c7a-9d25-c152a4022507 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:24.969606 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee5ff71-21ac-4c7a-9d25-c152a4022507 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:24.978245 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 225/445] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 2 19:00:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:24.979067 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee5ff71-21ac-4c7a-9d25-c152a4022507 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:25.027805 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:25.033567 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3269f0-4e76-4319-8dc8-b3bd781dafb5 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:25.068271 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c9b8d6-6b7d-4571-a3af-f54a87fe98cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:25.069136 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c9b8d6-6b7d-4571-a3af-f54a87fe98cd None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:25.069974 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c9b8d6-6b7d-4571-a3af-f54a87fe98cd None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:25.070251 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c9b8d6-6b7d-4571-a3af-f54a87fe98cd None admin] PATH_INFO: `/v3/groups/37365d258edc4d9b852736f631e192fe` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:25.071561 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c9b8d6-6b7d-4571-a3af-f54a87fe98cd None admin] RBAC: Authorizing `identity:get_group(group_id=37365d258edc4d9b852736f631e192fe)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:25.073015 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c9b8d6-6b7d-4571-a3af-f54a87fe98cd None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:25.074631 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 221/446] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Jun 2 19:00:24 2022] GET /identity/v3/groups/37365d258edc4d9b852736f631e192fe => generated 226 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:25.079665 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c9b8d6-6b7d-4571-a3af-f54a87fe98cd None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:25.082854 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee5ff71-21ac-4c7a-9d25-c152a4022507 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:25.116121 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93244d58-d534-4471-863b-5f5bb5305427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:25.116771 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93244d58-d534-4471-863b-5f5bb5305427 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:25.117059 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93244d58-d534-4471-863b-5f5bb5305427 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:25.117339 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93244d58-d534-4471-863b-5f5bb5305427 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:25.170062 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:25.175777 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c9b8d6-6b7d-4571-a3af-f54a87fe98cd None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:25.185922 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93244d58-d534-4471-863b-5f5bb5305427 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:25.187409 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93244d58-d534-4471-863b-5f5bb5305427 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:25.211675 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06451acd-76bf-4542-b261-232ada40f668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:25.212416 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06451acd-76bf-4542-b261-232ada40f668 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:25.212946 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06451acd-76bf-4542-b261-232ada40f668 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:25.213367 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06451acd-76bf-4542-b261-232ada40f668 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:25.235423 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:25.282373 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06451acd-76bf-4542-b261-232ada40f668 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:25.283879 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06451acd-76bf-4542-b261-232ada40f668 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:25.284864 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 226/447] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 2 19:00:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 02 19:00:25.285698 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93244d58-d534-4471-863b-5f5bb5305427 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:25.380376 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:25.386545 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93244d58-d534-4471-863b-5f5bb5305427 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:25.392381 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 222/448] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 2 19:00:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 02 19:00:25.392381 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06451acd-76bf-4542-b261-232ada40f668 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:25.425114 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a2a5ea-156f-4a76-9977-554238143112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:25.425766 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a2a5ea-156f-4a76-9977-554238143112 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:25.426048 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a2a5ea-156f-4a76-9977-554238143112 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:25.426333 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a2a5ea-156f-4a76-9977-554238143112 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:25.427182 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a2a5ea-156f-4a76-9977-554238143112 None admin] RBAC: Adding query filter params (group.id=37365d258edc4d9b852736f631e192fe, role.id=77d5301ed78146468d4115b512cc70b7, scope.project.id=5f19550a1b254991ba8eb40da4841fcf) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:25.427490 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a2a5ea-156f-4a76-9977-554238143112 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:25.428999 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a2a5ea-156f-4a76-9977-554238143112 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:25.434807 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 227/449] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Jun 2 19:00:25 2022] GET /identity/v3/role_assignments?group.id=37365d258edc4d9b852736f631e192fe&scope.project.id=5f19550a1b254991ba8eb40da4841fcf&role.id=77d5301ed78146468d4115b512cc70b7 => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:25.435439 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a2a5ea-156f-4a76-9977-554238143112 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:25.471453 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:25.496649 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06451acd-76bf-4542-b261-232ada40f668 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:25.536405 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0eb162-77d7-40a0-bd58-c6f875789468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:25.537396 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0eb162-77d7-40a0-bd58-c6f875789468 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:25.537852 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0eb162-77d7-40a0-bd58-c6f875789468 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:25.538899 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0eb162-77d7-40a0-bd58-c6f875789468 None admin] PATH_INFO: `/v3/projects/5f19550a1b254991ba8eb40da4841fcf` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:25.540515 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0eb162-77d7-40a0-bd58-c6f875789468 None admin] RBAC: Authorizing `identity:get_project(project_id=5f19550a1b254991ba8eb40da4841fcf)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:25.542508 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0eb162-77d7-40a0-bd58-c6f875789468 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:25.545207 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 223/450] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 2 19:00:25 2022] GET /identity/v3/projects/5f19550a1b254991ba8eb40da4841fcf => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:25.551855 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0eb162-77d7-40a0-bd58-c6f875789468 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:25.557593 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a2a5ea-156f-4a76-9977-554238143112 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:25.596425 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e6f3df-7453-4040-81cd-20e6dce64f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:25.597322 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e6f3df-7453-4040-81cd-20e6dce64f5e None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:25.597616 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e6f3df-7453-4040-81cd-20e6dce64f5e None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:25.597896 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e6f3df-7453-4040-81cd-20e6dce64f5e None admin] PATH_INFO: `/v3/projects/879ee62507da4a8282b2418099da56e5` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:25.599230 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e6f3df-7453-4040-81cd-20e6dce64f5e None admin] RBAC: Authorizing `identity:get_project(project_id=879ee62507da4a8282b2418099da56e5)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:25.601381 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e6f3df-7453-4040-81cd-20e6dce64f5e None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:25.603222 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 228/451] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 2 19:00:25 2022] GET /identity/v3/projects/879ee62507da4a8282b2418099da56e5 => generated 344 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:25.609012 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e6f3df-7453-4040-81cd-20e6dce64f5e None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:25.630793 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:25.673757 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0eb162-77d7-40a0-bd58-c6f875789468 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:25.702080 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:25.720423 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbd1189-b4ec-472d-b9fa-e0944cd7db6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:25.721333 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd1189-b4ec-472d-b9fa-e0944cd7db6f None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:25.721665 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd1189-b4ec-472d-b9fa-e0944cd7db6f None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:25.721895 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd1189-b4ec-472d-b9fa-e0944cd7db6f None admin] PATH_INFO: `/v3/groups/37365d258edc4d9b852736f631e192fe` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:25.723239 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbd1189-b4ec-472d-b9fa-e0944cd7db6f None admin] RBAC: Authorizing `identity:get_group(group_id=37365d258edc4d9b852736f631e192fe)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:25.724677 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e6f3df-7453-4040-81cd-20e6dce64f5e None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:25.724780 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbd1189-b4ec-472d-b9fa-e0944cd7db6f None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:25.727131 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 224/452] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Jun 2 19:00:25 2022] GET /identity/v3/groups/37365d258edc4d9b852736f631e192fe => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:00:25.732966 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbd1189-b4ec-472d-b9fa-e0944cd7db6f None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:25.771210 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d9eed5-b408-4bfe-94e4-e4be3f115cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:25.772097 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d9eed5-b408-4bfe-94e4-e4be3f115cdc None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:25.772909 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d9eed5-b408-4bfe-94e4-e4be3f115cdc None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:25.773199 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d9eed5-b408-4bfe-94e4-e4be3f115cdc None admin] PATH_INFO: `/v3/groups/f26ed4dcd678441abf63c4284a4cc43d` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:25.775434 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d9eed5-b408-4bfe-94e4-e4be3f115cdc None admin] RBAC: Authorizing `identity:get_group(group_id=f26ed4dcd678441abf63c4284a4cc43d)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:25.776889 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d9eed5-b408-4bfe-94e4-e4be3f115cdc None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:25.778530 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 229/453] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Jun 2 19:00:25 2022] GET /identity/v3/groups/f26ed4dcd678441abf63c4284a4cc43d => generated 229 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:00:25.784337 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d9eed5-b408-4bfe-94e4-e4be3f115cdc None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:25.857751 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbd1189-b4ec-472d-b9fa-e0944cd7db6f None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:25.887713 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:25.892984 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b69c7-8e68-497b-a906-af411bb656af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:25.893679 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b69c7-8e68-497b-a906-af411bb656af None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:25.893969 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b69c7-8e68-497b-a906-af411bb656af None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:25.894250 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b69c7-8e68-497b-a906-af411bb656af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:25.895145 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1b69c7-8e68-497b-a906-af411bb656af None admin] RBAC: Adding query filter params (group.id=37365d258edc4d9b852736f631e192fe, role.id=2e0c054b646e4ed2a0e7e5962f09a50e, scope.project.id=5f19550a1b254991ba8eb40da4841fcf) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:25.895466 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1b69c7-8e68-497b-a906-af411bb656af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:25.896918 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1b69c7-8e68-497b-a906-af411bb656af None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:25.898797 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d9eed5-b408-4bfe-94e4-e4be3f115cdc None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:25.903663 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 225/454] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Jun 2 19:00:25 2022] GET /identity/v3/role_assignments?group.id=37365d258edc4d9b852736f631e192fe&scope.project.id=5f19550a1b254991ba8eb40da4841fcf&role.id=2e0c054b646e4ed2a0e7e5962f09a50e => generated 611 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:00:25.934497 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4829fa7d-44b2-4a7d-9839-2aebdf36b939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:25.935215 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4829fa7d-44b2-4a7d-9839-2aebdf36b939 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:25.935505 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4829fa7d-44b2-4a7d-9839-2aebdf36b939 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:25.935784 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4829fa7d-44b2-4a7d-9839-2aebdf36b939 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:25.936645 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4829fa7d-44b2-4a7d-9839-2aebdf36b939 None admin] RBAC: Adding query filter params (group.id=f26ed4dcd678441abf63c4284a4cc43d, role.id=f13fdd0847924014abf1e95594195e78, scope.project.id=879ee62507da4a8282b2418099da56e5) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:25.936962 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4829fa7d-44b2-4a7d-9839-2aebdf36b939 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:25.938462 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4829fa7d-44b2-4a7d-9839-2aebdf36b939 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:25.945847 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 230/455] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Jun 2 19:00:25 2022] GET /identity/v3/role_assignments?group.id=f26ed4dcd678441abf63c4284a4cc43d&scope.project.id=879ee62507da4a8282b2418099da56e5&role.id=f13fdd0847924014abf1e95594195e78 => generated 611 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:00:28.729674 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e51b5-b55e-431a-97a7-a88ddc61c180 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:28.740176 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e51b5-b55e-431a-97a7-a88ddc61c180 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:28.740176 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e51b5-b55e-431a-97a7-a88ddc61c180 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:28.740176 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 226/456] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:00:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:00:28.740414 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb64f96-281a-4f57-a083-0b8b2fa41747 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:28.740414 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb64f96-281a-4f57-a083-0b8b2fa41747 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:28.740414 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb64f96-281a-4f57-a083-0b8b2fa41747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:28.767791 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-8fb64f96-281a-4f57-a083-0b8b2fa41747 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:00:28.768995 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-8fb64f96-281a-4f57-a083-0b8b2fa41747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:00:28.779760 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:28.849856 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 231/457] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:00:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 19:00:28.901278 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496e51b5-b55e-431a-97a7-a88ddc61c180 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:28.940527 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b234c6b-2156-4dde-a5fa-410280bd2711 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:28.941173 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b234c6b-2156-4dde-a5fa-410280bd2711 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:28.941651 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b234c6b-2156-4dde-a5fa-410280bd2711 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:28.943546 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 232/458] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:00:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:00:28.948974 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134488fc-f3c7-48af-8239-0960b6660de4 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:28.949503 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134488fc-f3c7-48af-8239-0960b6660de4 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:28.949992 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134488fc-f3c7-48af-8239-0960b6660de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:28.966794 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-134488fc-f3c7-48af-8239-0960b6660de4 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:00:28.968330 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-134488fc-f3c7-48af-8239-0960b6660de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:00:29.054403 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 233/459] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:00:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 02 19:00:29.096700 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496e51b5-b55e-431a-97a7-a88ddc61c180 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:29.136150 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24827f7-9c27-430c-918d-2ecb88c978cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:29.137223 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24827f7-9c27-430c-918d-2ecb88c978cd None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:29.137758 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24827f7-9c27-430c-918d-2ecb88c978cd None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:29.138241 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24827f7-9c27-430c-918d-2ecb88c978cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:29.151062 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134488fc-f3c7-48af-8239-0960b6660de4 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:29.154504 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24827f7-9c27-430c-918d-2ecb88c978cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:29.160639 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24827f7-9c27-430c-918d-2ecb88c978cd None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:29.193240 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-e24827f7-9c27-430c-918d-2ecb88c978cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:29.193240 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:29.193240 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:29.193240 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:29.193240 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:29.193240 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:29.193240 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:29.193240 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:29.193240 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:29.193240 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:29.193240 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:29.193240 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:29.193240 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:00:29.193240 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:00:29.193240 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:00:29.194187 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:00:29.194187 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:29.194187 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:29.194187 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:29.194187 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:29.194187 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:29.194187 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:29.194187 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:29.194187 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:29.194187 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:29.194187 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:29.194187 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:29.194187 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:29.194187 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:29.194187 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:29.194187 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:29.194187 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:29.195149 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:00:29.195149 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:00:29.195149 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:29.195149 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:00:29.205389 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 227/460] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:00:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:29.214959 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24827f7-9c27-430c-918d-2ecb88c978cd None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:29.286439 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:29.335810 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:29.394400 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134488fc-f3c7-48af-8239-0960b6660de4 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:29.433903 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4c0480-928e-4aa0-8f91-e07dc798391a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:29.435153 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c0480-928e-4aa0-8f91-e07dc798391a None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:29.435662 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c0480-928e-4aa0-8f91-e07dc798391a None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:29.436163 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c0480-928e-4aa0-8f91-e07dc798391a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:29.448699 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4c0480-928e-4aa0-8f91-e07dc798391a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:29.450634 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4c0480-928e-4aa0-8f91-e07dc798391a None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:29.470363 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-3e4c0480-928e-4aa0-8f91-e07dc798391a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:29.470363 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:29.470363 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:29.470363 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:29.470363 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:29.470363 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:29.470363 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:29.470363 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:29.470363 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:29.470363 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:29.470363 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:29.470363 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:29.470363 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:00:29.470363 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:00:29.470363 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:00:29.471349 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:00:29.471349 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:29.471349 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:29.471349 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:29.471349 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:29.471349 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:29.471349 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:29.471349 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:29.471349 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:29.471349 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:29.471349 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:29.471349 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:29.471349 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:29.471349 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:29.471349 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:29.471349 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:29.471349 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:29.472262 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:00:29.472262 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:00:29.472262 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:29.472262 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:00:29.473969 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 234/461] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:00:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:29.480045 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b068a-3c8f-403d-9f52-f148174e35c3 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:29.480341 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b068a-3c8f-403d-9f52-f148174e35c3 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:29.480626 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b068a-3c8f-403d-9f52-f148174e35c3 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:29.482138 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 235/462] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:00:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:00:29.483003 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f9bed-707d-4436-bc7e-63ad14fe0604 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:29.483563 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f9bed-707d-4436-bc7e-63ad14fe0604 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:29.484076 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f9bed-707d-4436-bc7e-63ad14fe0604 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:29.485719 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 236/463] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:00:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:00:29.486636 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4f9bed-707d-4436-bc7e-63ad14fe0604 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:29.509007 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24827f7-9c27-430c-918d-2ecb88c978cd None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:29.544597 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f28b3a-43b3-46f4-ab5f-1b1bd0ad7c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:29.545300 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f28b3a-43b3-46f4-ab5f-1b1bd0ad7c72 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:29.545590 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f28b3a-43b3-46f4-ab5f-1b1bd0ad7c72 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:29.545871 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f28b3a-43b3-46f4-ab5f-1b1bd0ad7c72 None admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:29.546607 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f28b3a-43b3-46f4-ab5f-1b1bd0ad7c72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:29.546951 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f28b3a-43b3-46f4-ab5f-1b1bd0ad7c72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:29.548507 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f28b3a-43b3-46f4-ab5f-1b1bd0ad7c72 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:29.560945 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 228/464] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:00:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:29.562015 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e4d31e-ac32-445a-8da0-50381ab1b3cc None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:29.562311 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e4d31e-ac32-445a-8da0-50381ab1b3cc None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:29.562592 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e4d31e-ac32-445a-8da0-50381ab1b3cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:29.579539 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-10e4d31e-ac32-445a-8da0-50381ab1b3cc None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:00:29.580741 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-10e4d31e-ac32-445a-8da0-50381ab1b3cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:00:29.618939 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4f9bed-707d-4436-bc7e-63ad14fe0604 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:29.650493 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 229/465] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:00:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 19:00:29.659314 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:29.660091 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb5aac4-762d-44aa-a8c3-8350dc6e05cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:29.660391 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813a223c-ff22-4ac3-9624-fcc5826b12d7 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:29.660685 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813a223c-ff22-4ac3-9624-fcc5826b12d7 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:29.660800 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb5aac4-762d-44aa-a8c3-8350dc6e05cf None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:29.660967 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813a223c-ff22-4ac3-9624-fcc5826b12d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:29.661068 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb5aac4-762d-44aa-a8c3-8350dc6e05cf None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:29.666768 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb5aac4-762d-44aa-a8c3-8350dc6e05cf None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:29.666768 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb5aac4-762d-44aa-a8c3-8350dc6e05cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:29.666768 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb5aac4-762d-44aa-a8c3-8350dc6e05cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:29.666768 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb5aac4-762d-44aa-a8c3-8350dc6e05cf None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:29.680769 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-813a223c-ff22-4ac3-9624-fcc5826b12d7 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:00:29.682251 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-813a223c-ff22-4ac3-9624-fcc5826b12d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:00:29.683467 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 237/466] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:00:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:29.686061 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb5aac4-762d-44aa-a8c3-8350dc6e05cf None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:29.713579 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:29.751338 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 230/467] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:00:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 19:00:29.752283 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813a223c-ff22-4ac3-9624-fcc5826b12d7 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:29.800558 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb5aac4-762d-44aa-a8c3-8350dc6e05cf None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:29.837240 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060cbdd7-f802-4ce0-9334-6895761bae91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:29.838221 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060cbdd7-f802-4ce0-9334-6895761bae91 None admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:29.838704 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060cbdd7-f802-4ce0-9334-6895761bae91 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:29.839032 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060cbdd7-f802-4ce0-9334-6895761bae91 None admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:29.839750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060cbdd7-f802-4ce0-9334-6895761bae91 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:29.841209 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060cbdd7-f802-4ce0-9334-6895761bae91 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:29.863039 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813a223c-ff22-4ac3-9624-fcc5826b12d7 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:29.896956 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac69b99-c9f0-4b5e-95a2-5c0b696687fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:29.897819 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac69b99-c9f0-4b5e-95a2-5c0b696687fd None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:29.898104 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac69b99-c9f0-4b5e-95a2-5c0b696687fd None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:29.898393 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac69b99-c9f0-4b5e-95a2-5c0b696687fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:29.910637 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac69b99-c9f0-4b5e-95a2-5c0b696687fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:29.912803 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac69b99-c9f0-4b5e-95a2-5c0b696687fd None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:29.930835 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-8ac69b99-c9f0-4b5e-95a2-5c0b696687fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:29.930835 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:29.930835 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:29.930835 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:29.930835 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:29.930835 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:29.930835 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:29.930835 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:29.930835 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:29.930835 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:29.930835 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:29.930835 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:29.930835 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:00:29.930835 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:00:29.930835 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:00:29.931771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:00:29.931771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:29.931771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:29.931771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:29.931771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:29.931771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:29.931771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:29.931771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:29.931771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:29.931771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:29.931771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:29.931771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:29.931771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:29.931771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:29.931771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:29.931771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:29.931771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:29.932631 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:00:29.932631 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:00:29.932631 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:29.932631 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:00:29.932631 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 231/468] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:00:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:29.932899 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac69b99-c9f0-4b5e-95a2-5c0b696687fd None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:29.940847 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 238/469] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 2 19:00:29 2022] POST /identity/v3/users => generated 246 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:00:29.950888 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060cbdd7-f802-4ce0-9334-6895761bae91 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:29.963639 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:29.999019 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:30.068638 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac69b99-c9f0-4b5e-95a2-5c0b696687fd None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:30.113439 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fcf228-624e-4729-9671-6ab93f29e191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:30.114632 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fcf228-624e-4729-9671-6ab93f29e191 None admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:30.115566 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fcf228-624e-4729-9671-6ab93f29e191 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:30.116126 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fcf228-624e-4729-9671-6ab93f29e191 None admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:30.117647 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fcf228-624e-4729-9671-6ab93f29e191 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:30.119494 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fcf228-624e-4729-9671-6ab93f29e191 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:30.215340 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 232/470] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 2 19:00:29 2022] POST /identity/v3/users => generated 248 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:00:30.216506 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fcf228-624e-4729-9671-6ab93f29e191 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:30.223587 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060cbdd7-f802-4ce0-9334-6895761bae91 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:30.280283 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d099436c-4a19-42f7-a033-28748a71675b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:30.281780 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099436c-4a19-42f7-a033-28748a71675b None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:30.283135 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099436c-4a19-42f7-a033-28748a71675b None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:30.283263 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099436c-4a19-42f7-a033-28748a71675b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:30.302838 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d099436c-4a19-42f7-a033-28748a71675b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:30.305668 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d099436c-4a19-42f7-a033-28748a71675b None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:30.328132 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-d099436c-4a19-42f7-a033-28748a71675b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:30.328132 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:30.328132 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:30.328132 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:30.328132 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:30.328132 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:30.328132 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:30.328132 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:30.328132 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:30.328132 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:30.328132 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:30.328132 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:30.328132 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:00:30.328132 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:00:30.328132 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:00:30.329083 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:00:30.329083 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:30.329083 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:30.329083 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:30.329083 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:30.329083 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:30.329083 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:30.329083 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:30.329083 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:30.329083 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:30.329083 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:30.329083 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:30.329083 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:30.329083 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:30.329083 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:30.329083 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:30.329083 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:30.330062 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:00:30.330062 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:00:30.330062 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:30.330062 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:00:30.330062 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 239/471] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:00:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 388 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:30.336873 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d099436c-4a19-42f7-a033-28748a71675b None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:30.341628 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:30.370204 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fcf228-624e-4729-9671-6ab93f29e191 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:30.410873 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2837826-5c37-4606-8a9a-107cc1404e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:30.411888 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2837826-5c37-4606-8a9a-107cc1404e2d None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:30.412369 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2837826-5c37-4606-8a9a-107cc1404e2d None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:30.412885 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2837826-5c37-4606-8a9a-107cc1404e2d None admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:30.414476 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2837826-5c37-4606-8a9a-107cc1404e2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:30.415060 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2837826-5c37-4606-8a9a-107cc1404e2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:30.416741 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2837826-5c37-4606-8a9a-107cc1404e2d None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:30.432957 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 233/472] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:00:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:30.439156 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2837826-5c37-4606-8a9a-107cc1404e2d None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:30.445170 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:30.452035 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d099436c-4a19-42f7-a033-28748a71675b None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:30.488198 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e314ccd9-64d2-4a1d-ac2c-5bfdc9663522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:30.489313 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e314ccd9-64d2-4a1d-ac2c-5bfdc9663522 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:30.489669 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e314ccd9-64d2-4a1d-ac2c-5bfdc9663522 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:30.490018 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e314ccd9-64d2-4a1d-ac2c-5bfdc9663522 None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:30.490776 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e314ccd9-64d2-4a1d-ac2c-5bfdc9663522 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:30.491087 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e314ccd9-64d2-4a1d-ac2c-5bfdc9663522 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:30.493112 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e314ccd9-64d2-4a1d-ac2c-5bfdc9663522 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:30.506769 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 240/473] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:00:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:00:30.512307 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e314ccd9-64d2-4a1d-ac2c-5bfdc9663522 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:30.551230 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2837826-5c37-4606-8a9a-107cc1404e2d None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:30.590276 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c07b09-7784-4b19-abaf-6ae87e24ccf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:30.590998 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c07b09-7784-4b19-abaf-6ae87e24ccf4 None admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:30.591285 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c07b09-7784-4b19-abaf-6ae87e24ccf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:30.591564 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c07b09-7784-4b19-abaf-6ae87e24ccf4 None admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:30.592410 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c07b09-7784-4b19-abaf-6ae87e24ccf4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:30.593889 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c07b09-7784-4b19-abaf-6ae87e24ccf4 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:30.603576 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:30.618949 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e314ccd9-64d2-4a1d-ac2c-5bfdc9663522 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:30.656456 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9dba4d-aaca-4c0a-a959-cdac672eb482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:30.657134 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9dba4d-aaca-4c0a-a959-cdac672eb482 None admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:30.657464 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9dba4d-aaca-4c0a-a959-cdac672eb482 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:30.657794 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9dba4d-aaca-4c0a-a959-cdac672eb482 None admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:30.658506 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9dba4d-aaca-4c0a-a959-cdac672eb482 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:30.659987 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9dba4d-aaca-4c0a-a959-cdac672eb482 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:30.710863 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 234/474] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 2 19:00:30 2022] POST /identity/v3/users => generated 248 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 02 19:00:30.712956 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ac653-c9ba-437a-b18c-8eb1a33ca04d None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:30.713413 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ac653-c9ba-437a-b18c-8eb1a33ca04d None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:30.713850 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ac653-c9ba-437a-b18c-8eb1a33ca04d None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:30.715749 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 235/475] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:00:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:00:30.724388 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b47a3d8-628b-4c70-bb8f-3425df7197cb None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:30.724729 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b47a3d8-628b-4c70-bb8f-3425df7197cb None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:30.724958 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b47a3d8-628b-4c70-bb8f-3425df7197cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:30.742032 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 241/476] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 2 19:00:30 2022] POST /identity/v3/users => generated 249 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 02 19:00:30.763958 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-6b47a3d8-628b-4c70-bb8f-3425df7197cb None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:00:30.765194 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-6b47a3d8-628b-4c70-bb8f-3425df7197cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:00:30.872405 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 236/477] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:00:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 19:00:30.912552 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9dba4d-aaca-4c0a-a959-cdac672eb482 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:30.930711 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:31.040553 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9dba4d-aaca-4c0a-a959-cdac672eb482 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:31.079392 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f614c7e1-b2a0-44fe-b3de-c46b228e888a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:31.080065 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f614c7e1-b2a0-44fe-b3de-c46b228e888a None admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:31.080349 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f614c7e1-b2a0-44fe-b3de-c46b228e888a None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:31.080625 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f614c7e1-b2a0-44fe-b3de-c46b228e888a None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:31.081382 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f614c7e1-b2a0-44fe-b3de-c46b228e888a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:31.082781 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f614c7e1-b2a0-44fe-b3de-c46b228e888a None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:31.122556 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.sql.core [None req-f614c7e1-b2a0-44fe-b3de-c46b228e888a None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 02 19:00:31.122556 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 02 19:00:31.122556 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [parameters: {'id': 'dc56448732a84e6c991bf25f7231ffa3', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 02 19:00:31.122556 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80491) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 02 19:00:31.140487 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-f614c7e1-b2a0-44fe-b3de-c46b228e888a None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 02 19:00:31.140487 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:31.140487 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 02 19:00:31.140487 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 02 19:00:31.140487 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 02 19:00:31.140487 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 02 19:00:31.140487 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 02 19:00:31.140487 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application result = self._query(query) Jun 02 19:00:31.140487 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 02 19:00:31.140487 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application conn.query(q) Jun 02 19:00:31.140487 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 02 19:00:31.140487 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 02 19:00:31.140487 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 02 19:00:31.140487 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application result.read() Jun 02 19:00:31.140487 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 02 19:00:31.141471 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 02 19:00:31.141471 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 02 19:00:31.141471 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 02 19:00:31.141471 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 02 19:00:31.141471 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 02 19:00:31.141471 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 02 19:00:31.141471 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 02 19:00:31.141471 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 02 19:00:31.141471 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:00:31.141471 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 02 19:00:31.141471 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:00:31.141471 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:31.141471 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 02 19:00:31.141471 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 02 19:00:31.141471 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 02 19:00:31.141471 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return ref.to_dict() Jun 02 19:00:31.141471 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 02 19:00:31.141471 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application next(self.gen) Jun 02 19:00:31.142852 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 02 19:00:31.142852 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application yield resource Jun 02 19:00:31.142852 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 02 19:00:31.142852 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application next(self.gen) Jun 02 19:00:31.142852 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 02 19:00:31.142852 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self.session.rollback() Jun 02 19:00:31.142852 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 02 19:00:31.142852 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self.force_reraise() Jun 02 19:00:31.142852 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 02 19:00:31.142852 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise self.value Jun 02 19:00:31.142852 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 02 19:00:31.142852 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 02 19:00:31.142852 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 02 19:00:31.142852 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application session.commit() Jun 02 19:00:31.142852 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit Jun 02 19:00:31.142852 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 02 19:00:31.142852 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 02 19:00:31.143760 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._prepare_impl() Jun 02 19:00:31.143760 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 02 19:00:31.143760 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self.session.flush() Jun 02 19:00:31.143760 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush Jun 02 19:00:31.143760 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._flush(objects) Jun 02 19:00:31.143760 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush Jun 02 19:00:31.143760 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 02 19:00:31.143760 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 02 19:00:31.143760 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application compat.raise_( Jun 02 19:00:31.143760 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ Jun 02 19:00:31.143760 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception Jun 02 19:00:31.143760 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush Jun 02 19:00:31.143760 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application flush_context.execute() Jun 02 19:00:31.143760 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 02 19:00:31.143760 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rec.execute(self) Jun 02 19:00:31.143760 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 02 19:00:31.143760 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 02 19:00:31.144644 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 02 19:00:31.144644 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 02 19:00:31.144644 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 02 19:00:31.144644 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 02 19:00:31.144644 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 02 19:00:31.144644 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 02 19:00:31.144644 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 02 19:00:31.144644 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 02 19:00:31.144644 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 02 19:00:31.144644 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 02 19:00:31.144644 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 02 19:00:31.144644 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 02 19:00:31.144644 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 02 19:00:31.144644 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 02 19:00:31.144644 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ Jun 02 19:00:31.144644 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception Jun 02 19:00:31.145490 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 02 19:00:31.145490 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 02 19:00:31.145490 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 02 19:00:31.145490 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 02 19:00:31.145490 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 02 19:00:31.145490 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application result = self._query(query) Jun 02 19:00:31.145490 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 02 19:00:31.145490 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application conn.query(q) Jun 02 19:00:31.145490 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 02 19:00:31.145490 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 02 19:00:31.145490 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 02 19:00:31.145490 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application result.read() Jun 02 19:00:31.145490 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 02 19:00:31.145490 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 02 19:00:31.145490 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 02 19:00:31.145490 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 02 19:00:31.148887 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 02 19:00:31.148887 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 02 19:00:31.148887 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 02 19:00:31.148887 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 02 19:00:31.148887 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 02 19:00:31.148887 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 02 19:00:31.148887 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [parameters: {'id': 'dc56448732a84e6c991bf25f7231ffa3', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 02 19:00:31.148887 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 02 19:00:31.148887 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:00:31.148887 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 02 19:00:31.148887 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:00:31.148887 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:31.148887 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:31.148887 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:31.148887 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:31.148887 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:31.150936 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:31.150936 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:31.150936 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:31.150936 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:31.150936 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:31.150936 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:31.150936 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 02 19:00:31.150936 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 02 19:00:31.150936 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:31.150936 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:31.150936 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 02 19:00:31.150936 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 02 19:00:31.150936 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 02 19:00:31.150936 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 02 19:00:31.150936 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 02 19:00:31.150936 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:00:31.152928 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 242/478] 10.222.0.33 () {60 vars in 1131 bytes} [Thu Jun 2 19:00:30 2022] POST /identity/v3/roles => generated 144 bytes in 230 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:31.159232 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b47a3d8-628b-4c70-bb8f-3425df7197cb None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:31.272822 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b47a3d8-628b-4c70-bb8f-3425df7197cb None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:31.309685 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:31.322287 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a643eb89-7cb9-4ed1-b27b-c05e6bc7eeff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:31.325797 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a643eb89-7cb9-4ed1-b27b-c05e6bc7eeff None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:31.325797 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a643eb89-7cb9-4ed1-b27b-c05e6bc7eeff None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:31.325797 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a643eb89-7cb9-4ed1-b27b-c05e6bc7eeff None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:31.339634 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a643eb89-7cb9-4ed1-b27b-c05e6bc7eeff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:00:31.351718 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-a643eb89-7cb9-4ed1-b27b-c05e6bc7eeff None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 02 19:00:31.351718 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:31.351718 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:31.351718 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:31.351718 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:31.351718 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:31.351718 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:31.351718 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:31.351718 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:31.351718 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:31.351718 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:31.351718 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:31.351718 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:00:31.351718 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:00:31.351718 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:00:31.351718 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:00:31.352644 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:00:31.352644 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:00:31.352644 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:00:31.352644 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:00:31.352644 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:31.352644 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:31.352644 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:31.352644 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:31.352644 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:31.352644 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:31.352644 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:31.352644 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:31.352644 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:31.352644 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:31.352644 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:31.352644 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:31.353763 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:31.353763 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:31.353763 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:31.353763 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:31.353763 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:00:31.353763 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:00:31.353763 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:00:31.353763 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:00:31.353763 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:00:31.353763 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:00:31.353763 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 02 19:00:31.353763 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:00:31.353763 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 237/479] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 2 19:00:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:31.360307 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f614c7e1-b2a0-44fe-b3de-c46b228e888a None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:31.464823 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:31.492844 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f614c7e1-b2a0-44fe-b3de-c46b228e888a None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:31.535471 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fd6e21-13e7-42de-828f-b2eb079ae83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:31.536161 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd6e21-13e7-42de-828f-b2eb079ae83e None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:31.536513 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd6e21-13e7-42de-828f-b2eb079ae83e None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:31.536791 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd6e21-13e7-42de-828f-b2eb079ae83e None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:31.537566 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fd6e21-13e7-42de-828f-b2eb079ae83e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:31.537889 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fd6e21-13e7-42de-828f-b2eb079ae83e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:31.539924 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fd6e21-13e7-42de-828f-b2eb079ae83e None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:31.553982 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 243/480] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 2 19:00:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:32.868495 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566e9b6e-5709-4b6b-9175-fd5dab267b2b None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:32.869008 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566e9b6e-5709-4b6b-9175-fd5dab267b2b None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:32.869117 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566e9b6e-5709-4b6b-9175-fd5dab267b2b None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:32.870423 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 238/481] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:00:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:00:32.875761 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7541d26-b473-4ba0-8e25-7985802abbbe None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:32.876593 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7541d26-b473-4ba0-8e25-7985802abbbe None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:32.877071 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7541d26-b473-4ba0-8e25-7985802abbbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:32.900713 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-a7541d26-b473-4ba0-8e25-7985802abbbe None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:00:32.902239 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-a7541d26-b473-4ba0-8e25-7985802abbbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:00:32.991197 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 244/482] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:00:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 19:00:33.032007 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566e9b6e-5709-4b6b-9175-fd5dab267b2b None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:33.083676 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad0ff9-caa9-44ff-beb1-54e261b9d772 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:33.085173 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad0ff9-caa9-44ff-beb1-54e261b9d772 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:33.085660 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad0ff9-caa9-44ff-beb1-54e261b9d772 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:33.087922 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 245/483] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:00:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:00:33.094194 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb99d5b-7882-4d22-85b0-b0fd04c057e8 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:33.094475 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb99d5b-7882-4d22-85b0-b0fd04c057e8 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:33.095065 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb99d5b-7882-4d22-85b0-b0fd04c057e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:33.139190 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-5fb99d5b-7882-4d22-85b0-b0fd04c057e8 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:00:33.140795 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-5fb99d5b-7882-4d22-85b0-b0fd04c057e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:00:33.175890 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566e9b6e-5709-4b6b-9175-fd5dab267b2b None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:33.230785 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:33.232518 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ae5d4c-5663-4bea-ab78-64a35b7423ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:33.232518 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae5d4c-5663-4bea-ab78-64a35b7423ca None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:33.232518 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae5d4c-5663-4bea-ab78-64a35b7423ca None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:33.232518 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae5d4c-5663-4bea-ab78-64a35b7423ca None admin] PATH_INFO: `/v3/roles/service` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:33.245378 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 246/484] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:00:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 19:00:33.246196 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ae5d4c-5663-4bea-ab78-64a35b7423ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:00:33.254526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-d2ae5d4c-5663-4bea-ab78-64a35b7423ca None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 02 19:00:33.254526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:33.254526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:33.254526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:33.254526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:33.254526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:33.254526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:33.254526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:33.254526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:33.254526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:33.254526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:33.254526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:33.254526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:00:33.254526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:00:33.254526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:00:33.254526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:00:33.255551 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:00:33.255551 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:00:33.255551 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:00:33.255551 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:00:33.255551 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:33.255551 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:33.255551 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:33.255551 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:33.255551 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:33.255551 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:33.255551 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:33.255551 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:33.255551 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:33.255551 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:33.255551 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:33.255551 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:33.256474 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:33.256474 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:33.256474 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:33.256474 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:33.256474 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:00:33.256474 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:00:33.256474 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:00:33.256474 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:00:33.256474 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:00:33.256474 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:00:33.256474 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 02 19:00:33.256474 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:00:33.256474 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 239/485] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 2 19:00:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:33.261619 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb99d5b-7882-4d22-85b0-b0fd04c057e8 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:33.291433 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ae5d4c-5663-4bea-ab78-64a35b7423ca None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:33.376695 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:33.390180 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb99d5b-7882-4d22-85b0-b0fd04c057e8 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:33.425693 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b73b98-b7f9-4bf9-bfdc-a05d01d45d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:33.426729 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b73b98-b7f9-4bf9-bfdc-a05d01d45d5e None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:33.427315 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b73b98-b7f9-4bf9-bfdc-a05d01d45d5e None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:33.427791 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b73b98-b7f9-4bf9-bfdc-a05d01d45d5e None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:33.428748 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b73b98-b7f9-4bf9-bfdc-a05d01d45d5e None admin] RBAC: Adding query filter params (name=service) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:33.429275 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b73b98-b7f9-4bf9-bfdc-a05d01d45d5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:33.430914 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b73b98-b7f9-4bf9-bfdc-a05d01d45d5e None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:33.439904 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 247/486] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 2 19:00:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:33.448078 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b73b98-b7f9-4bf9-bfdc-a05d01d45d5e None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:33.500207 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ae5d4c-5663-4bea-ab78-64a35b7423ca None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:33.548428 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a7ecc6-3f89-4791-9b79-a462f6d2d812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:33.549304 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a7ecc6-3f89-4791-9b79-a462f6d2d812 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:33.549595 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a7ecc6-3f89-4791-9b79-a462f6d2d812 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:33.549870 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a7ecc6-3f89-4791-9b79-a462f6d2d812 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:33.562884 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:33.567688 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a7ecc6-3f89-4791-9b79-a462f6d2d812 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:00:33.575569 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-d5a7ecc6-3f89-4791-9b79-a462f6d2d812 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 02 19:00:33.575569 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:33.575569 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:33.575569 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:33.575569 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:33.575569 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:33.575569 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:33.575569 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:33.575569 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:33.575569 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:33.575569 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:33.575569 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:33.575569 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:00:33.575569 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:00:33.575569 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:00:33.575569 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:00:33.576585 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:00:33.576585 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:00:33.576585 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:00:33.576585 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:00:33.576585 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:33.576585 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:33.576585 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:33.576585 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:33.576585 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:33.576585 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:33.576585 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:33.576585 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:33.576585 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:33.576585 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:33.576585 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:33.576585 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:33.577340 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:33.577340 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:33.577340 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:33.577340 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:33.577340 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:00:33.577340 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:00:33.577340 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:00:33.577340 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:00:33.577340 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:00:33.577340 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:00:33.577340 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 02 19:00:33.577340 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:00:33.577340 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 240/487] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 2 19:00:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:33.588563 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a7ecc6-3f89-4791-9b79-a462f6d2d812 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:33.615069 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b73b98-b7f9-4bf9-bfdc-a05d01d45d5e None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:33.655861 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe623636-a86b-45f8-a2db-591fa13298d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:33.657509 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe623636-a86b-45f8-a2db-591fa13298d1 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:33.657940 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe623636-a86b-45f8-a2db-591fa13298d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:33.658356 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe623636-a86b-45f8-a2db-591fa13298d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:33.672789 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe623636-a86b-45f8-a2db-591fa13298d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:33.675325 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe623636-a86b-45f8-a2db-591fa13298d1 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:33.698232 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-fe623636-a86b-45f8-a2db-591fa13298d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:33.698232 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:33.698232 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:33.698232 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:33.698232 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:33.698232 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:33.698232 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:33.698232 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:33.698232 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:33.698232 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:33.698232 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:33.698232 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:33.698232 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:00:33.698232 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:00:33.698232 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:00:33.699400 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:00:33.699400 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:33.699400 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:33.699400 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:33.699400 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:33.699400 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:33.699400 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:33.699400 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:33.699400 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:33.699400 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:33.699400 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:33.699400 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:33.699400 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:33.699400 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:33.699400 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:33.699400 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:33.699400 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:33.700399 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:00:33.700399 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:00:33.700399 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:33.700399 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:00:33.702882 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 248/488] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:00:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:33.704482 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d7eab-aee7-491b-9cc0-888ba9c23f42 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:33.705308 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d7eab-aee7-491b-9cc0-888ba9c23f42 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:33.705750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d7eab-aee7-491b-9cc0-888ba9c23f42 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:33.707478 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 249/489] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:00:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:00:33.713627 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a768f604-3dd4-4b14-90a1-730a3855f817 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:33.713922 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a768f604-3dd4-4b14-90a1-730a3855f817 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:33.714203 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a768f604-3dd4-4b14-90a1-730a3855f817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:33.728960 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a7ecc6-3f89-4791-9b79-a462f6d2d812 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:33.731485 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:33.743213 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-a768f604-3dd4-4b14-90a1-730a3855f817 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:00:33.744944 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-a768f604-3dd4-4b14-90a1-730a3855f817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:00:33.777260 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff56bc0c-a23a-42cd-83fa-d16e496d1654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:33.777936 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff56bc0c-a23a-42cd-83fa-d16e496d1654 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:33.778222 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff56bc0c-a23a-42cd-83fa-d16e496d1654 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:33.778502 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff56bc0c-a23a-42cd-83fa-d16e496d1654 None admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:33.779469 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff56bc0c-a23a-42cd-83fa-d16e496d1654 None admin] RBAC: Adding query filter params (name=service) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:33.779868 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff56bc0c-a23a-42cd-83fa-d16e496d1654 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:33.781919 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff56bc0c-a23a-42cd-83fa-d16e496d1654 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:33.796818 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 241/490] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 2 19:00:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:33.796818 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff56bc0c-a23a-42cd-83fa-d16e496d1654 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:33.834054 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 250/491] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:00:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 02 19:00:33.835064 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a768f604-3dd4-4b14-90a1-730a3855f817 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:33.901752 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:33.911109 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff56bc0c-a23a-42cd-83fa-d16e496d1654 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:33.948777 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35762e90-57a4-4631-8af0-d99cada1d882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:33.950455 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35762e90-57a4-4631-8af0-d99cada1d882 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:33.951112 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35762e90-57a4-4631-8af0-d99cada1d882 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:33.951573 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35762e90-57a4-4631-8af0-d99cada1d882 None admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:33.952507 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35762e90-57a4-4631-8af0-d99cada1d882 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:33.952984 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35762e90-57a4-4631-8af0-d99cada1d882 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:33.954906 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35762e90-57a4-4631-8af0-d99cada1d882 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:33.965992 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a768f604-3dd4-4b14-90a1-730a3855f817 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:33.969147 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 242/492] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:00:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:33.970307 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35762e90-57a4-4631-8af0-d99cada1d882 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:33.999431 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:34.010655 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b9b42c-02f1-413b-8005-0c56fb65fc07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:34.011472 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b9b42c-02f1-413b-8005-0c56fb65fc07 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:34.011737 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b9b42c-02f1-413b-8005-0c56fb65fc07 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:34.012014 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b9b42c-02f1-413b-8005-0c56fb65fc07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:34.024153 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b9b42c-02f1-413b-8005-0c56fb65fc07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:34.025955 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b9b42c-02f1-413b-8005-0c56fb65fc07 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:34.045892 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-65b9b42c-02f1-413b-8005-0c56fb65fc07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:34.045892 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:34.045892 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:34.045892 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:34.045892 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:34.045892 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:34.045892 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:34.045892 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:34.045892 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:34.045892 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:34.045892 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:34.045892 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:34.045892 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:00:34.045892 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:00:34.045892 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:00:34.046923 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:00:34.046923 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:34.046923 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:34.046923 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:34.046923 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:34.046923 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:34.046923 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:34.046923 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:34.046923 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:34.046923 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:34.046923 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:34.046923 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:34.046923 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:34.046923 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:34.046923 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:34.046923 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:34.046923 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:34.047761 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:00:34.047761 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:00:34.047761 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:34.047761 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:00:34.050923 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 251/493] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:00:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:34.052359 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b9b42c-02f1-413b-8005-0c56fb65fc07 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:34.170698 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35762e90-57a4-4631-8af0-d99cada1d882 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:34.182039 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:34.191668 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b9b42c-02f1-413b-8005-0c56fb65fc07 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:34.216786 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cad0a2d-8891-48ea-abe1-02a5e838fe51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:34.217590 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad0a2d-8891-48ea-abe1-02a5e838fe51 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:34.217876 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad0a2d-8891-48ea-abe1-02a5e838fe51 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:34.218153 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad0a2d-8891-48ea-abe1-02a5e838fe51 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:34.227944 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901d935b-e82d-4e74-a17c-e79745cbeb55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:34.228752 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901d935b-e82d-4e74-a17c-e79745cbeb55 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:34.229024 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cad0a2d-8891-48ea-abe1-02a5e838fe51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:00:34.229132 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901d935b-e82d-4e74-a17c-e79745cbeb55 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:34.229305 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901d935b-e82d-4e74-a17c-e79745cbeb55 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:34.236582 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901d935b-e82d-4e74-a17c-e79745cbeb55 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:34.236777 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-6cad0a2d-8891-48ea-abe1-02a5e838fe51 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 02 19:00:34.236777 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:34.236777 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:34.236777 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:34.236777 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:34.236777 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:34.236777 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:34.236777 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:34.236777 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:34.236777 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:34.236777 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:34.236777 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:34.236777 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:00:34.236777 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:00:34.236777 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:00:34.236777 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:00:34.237617 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:00:34.237617 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:00:34.237617 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:00:34.237617 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:00:34.237617 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:34.237617 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:34.237617 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:34.237617 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:34.237617 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:34.237617 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:34.237617 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:34.237617 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:34.237617 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:34.237617 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:34.237617 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:34.237617 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:34.238432 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:34.238432 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:34.238432 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:34.238432 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:34.238432 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:00:34.238432 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:00:34.238432 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:00:34.238432 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:00:34.238432 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:00:34.238432 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:00:34.238432 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 02 19:00:34.238432 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:00:34.239854 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901d935b-e82d-4e74-a17c-e79745cbeb55 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:34.242863 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 243/494] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 2 19:00:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:34.243534 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cad0a2d-8891-48ea-abe1-02a5e838fe51 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:34.255272 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-901d935b-e82d-4e74-a17c-e79745cbeb55 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 02 19:00:34.255272 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:34.255272 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:34.255272 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:34.255272 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:34.255272 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:34.255272 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:34.255272 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:34.255272 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:34.255272 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:34.255272 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:34.255272 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:34.255272 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 02 19:00:34.255272 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 02 19:00:34.255272 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 02 19:00:34.255272 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 02 19:00:34.256244 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:34.256244 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:34.256244 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 02 19:00:34.256244 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:00:34.256244 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 02 19:00:34.256244 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:00:34.256244 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:34.256244 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:34.256244 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:34.256244 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:34.256244 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:34.256244 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:34.256244 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:34.256244 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:34.256244 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:34.256244 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:34.256244 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:34.256244 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:34.257822 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:34.257822 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:34.257822 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 02 19:00:34.257822 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 02 19:00:34.257822 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 02 19:00:34.257822 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 02 19:00:34.257822 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 02 19:00:34.257822 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 02 19:00:34.257822 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 02 19:00:34.257822 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:00:34.257822 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 252/495] 10.222.0.33 () {56 vars in 1093 bytes} [Thu Jun 2 19:00:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:34.258382 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9bc099-1501-4b5e-ad9a-a3a416b20880 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:34.259013 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9bc099-1501-4b5e-ad9a-a3a416b20880 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:34.259013 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9bc099-1501-4b5e-ad9a-a3a416b20880 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:34.260403 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 253/496] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:00:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:00:34.262410 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9bc099-1501-4b5e-ad9a-a3a416b20880 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:34.354459 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:34.373753 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cad0a2d-8891-48ea-abe1-02a5e838fe51 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:34.374196 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9bc099-1501-4b5e-ad9a-a3a416b20880 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:34.412213 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1296b3c7-2aed-4573-bc0e-34b9da1f409c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:34.412591 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82beee9a-0e46-4649-8bb6-ec4498cb8881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:34.412889 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1296b3c7-2aed-4573-bc0e-34b9da1f409c None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:34.413177 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1296b3c7-2aed-4573-bc0e-34b9da1f409c None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:34.413310 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82beee9a-0e46-4649-8bb6-ec4498cb8881 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:34.413460 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1296b3c7-2aed-4573-bc0e-34b9da1f409c None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:34.413607 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82beee9a-0e46-4649-8bb6-ec4498cb8881 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:34.413888 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82beee9a-0e46-4649-8bb6-ec4498cb8881 None admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:34.414208 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1296b3c7-2aed-4573-bc0e-34b9da1f409c None admin] RBAC: Adding query filter params (name=service) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:34.414520 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1296b3c7-2aed-4573-bc0e-34b9da1f409c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:34.414614 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82beee9a-0e46-4649-8bb6-ec4498cb8881 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:34.414940 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82beee9a-0e46-4649-8bb6-ec4498cb8881 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:34.415947 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1296b3c7-2aed-4573-bc0e-34b9da1f409c None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:34.416366 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82beee9a-0e46-4649-8bb6-ec4498cb8881 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:34.423903 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 254/497] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 2 19:00:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:34.424644 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1296b3c7-2aed-4573-bc0e-34b9da1f409c None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:34.427761 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 244/498] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:00:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:34.429213 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044c8e3c-257e-4154-9509-ef13c554e9d8 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:34.429505 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044c8e3c-257e-4154-9509-ef13c554e9d8 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:34.429786 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044c8e3c-257e-4154-9509-ef13c554e9d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:34.441406 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:34.448306 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-044c8e3c-257e-4154-9509-ef13c554e9d8 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:00:34.449547 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-044c8e3c-257e-4154-9509-ef13c554e9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:00:34.523601 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 245/499] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:00:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 19:00:34.525047 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1296b3c7-2aed-4573-bc0e-34b9da1f409c None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:34.525393 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2504e963-0f46-4647-847a-3e2259709b83 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:34.525876 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2504e963-0f46-4647-847a-3e2259709b83 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:34.526339 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2504e963-0f46-4647-847a-3e2259709b83 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:34.527830 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 246/500] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:00:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:00:34.528520 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2504e963-0f46-4647-847a-3e2259709b83 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:34.570181 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026005d3-5e0e-428b-90c6-66f199e4768d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:34.570903 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026005d3-5e0e-428b-90c6-66f199e4768d None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:34.571190 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026005d3-5e0e-428b-90c6-66f199e4768d None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:34.571466 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026005d3-5e0e-428b-90c6-66f199e4768d None admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:34.572729 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026005d3-5e0e-428b-90c6-66f199e4768d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:34.573059 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026005d3-5e0e-428b-90c6-66f199e4768d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:34.574918 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026005d3-5e0e-428b-90c6-66f199e4768d None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:34.606615 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:34.612175 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 255/501] 10.222.0.33 () {56 vars in 1133 bytes} [Thu Jun 2 19:00:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:34.613021 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026005d3-5e0e-428b-90c6-66f199e4768d None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:34.631869 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2504e963-0f46-4647-847a-3e2259709b83 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:34.673563 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d370317-4c48-4a0a-9192-74b71497487e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:34.674550 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d370317-4c48-4a0a-9192-74b71497487e None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:34.674992 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d370317-4c48-4a0a-9192-74b71497487e None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:34.675435 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d370317-4c48-4a0a-9192-74b71497487e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:34.687704 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d370317-4c48-4a0a-9192-74b71497487e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:34.689340 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d370317-4c48-4a0a-9192-74b71497487e None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:34.693788 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:34.708476 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-6d370317-4c48-4a0a-9192-74b71497487e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:34.708476 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:34.708476 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:34.708476 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:34.708476 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:34.708476 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:34.708476 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:34.708476 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:34.708476 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:34.708476 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:34.708476 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:34.708476 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:34.708476 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:00:34.708476 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:00:34.708476 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:00:34.709510 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:00:34.709510 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:34.709510 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:34.709510 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:34.709510 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:34.709510 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:34.709510 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:34.709510 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:34.709510 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:34.709510 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:34.709510 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:34.709510 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:34.709510 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:34.709510 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:34.709510 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:34.709510 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:34.709510 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:34.710685 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:00:34.710685 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:00:34.710685 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:34.710685 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:00:34.710685 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 247/502] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:00:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:34.711069 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23f4fba-1e6f-478d-b128-9f7093acb065 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:34.711244 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23f4fba-1e6f-478d-b128-9f7093acb065 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:34.711523 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23f4fba-1e6f-478d-b128-9f7093acb065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:34.727485 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-f23f4fba-1e6f-478d-b128-9f7093acb065 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:00:34.728658 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-f23f4fba-1e6f-478d-b128-9f7093acb065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:00:34.728865 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026005d3-5e0e-428b-90c6-66f199e4768d None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:34.763591 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c5846-d9cb-41fe-8b8f-a2d5d96c924f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:34.764394 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c5846-d9cb-41fe-8b8f-a2d5d96c924f None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:34.764682 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c5846-d9cb-41fe-8b8f-a2d5d96c924f None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:34.764961 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c5846-d9cb-41fe-8b8f-a2d5d96c924f None admin] PATH_INFO: `/v3/users/glance` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:34.772057 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7c5846-d9cb-41fe-8b8f-a2d5d96c924f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:34.773540 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7c5846-d9cb-41fe-8b8f-a2d5d96c924f None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:34.788126 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-3d7c5846-d9cb-41fe-8b8f-a2d5d96c924f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 02 19:00:34.788126 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:34.788126 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:34.788126 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:34.788126 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:34.788126 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:34.788126 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:34.788126 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:34.788126 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:34.788126 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:34.788126 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:34.788126 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:34.788126 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 02 19:00:34.788126 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 02 19:00:34.788126 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 02 19:00:34.788126 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 02 19:00:34.789194 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:34.789194 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:34.789194 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 02 19:00:34.789194 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:00:34.789194 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 02 19:00:34.789194 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:00:34.789194 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:34.789194 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:34.789194 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:34.789194 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:34.789194 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:34.789194 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:34.789194 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:34.789194 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:34.789194 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:34.789194 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:34.789194 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:34.789194 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:34.790082 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:34.790082 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:34.790082 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 02 19:00:34.790082 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 02 19:00:34.790082 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 02 19:00:34.790082 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 02 19:00:34.790082 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 02 19:00:34.790082 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 02 19:00:34.790082 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 02 19:00:34.790082 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:00:34.790082 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 256/503] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 2 19:00:34 2022] GET /identity/v3/users/glance => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:34.790639 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c5846-d9cb-41fe-8b8f-a2d5d96c924f None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:34.796589 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 248/504] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:00:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 19:00:34.797432 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23f4fba-1e6f-478d-b128-9f7093acb065 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:34.854523 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:34.889095 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c5846-d9cb-41fe-8b8f-a2d5d96c924f None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:34.897954 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23f4fba-1e6f-478d-b128-9f7093acb065 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:34.918979 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:34.924088 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b75af1-20b2-40f3-b0e7-d2a78e4a46ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:34.928204 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b75af1-20b2-40f3-b0e7-d2a78e4a46ba None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:34.928204 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b75af1-20b2-40f3-b0e7-d2a78e4a46ba None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:34.928204 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b75af1-20b2-40f3-b0e7-d2a78e4a46ba None admin] PATH_INFO: `/v3/roles/service` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:34.932646 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e0f496-fb52-45b0-855a-4eaecf53429f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:34.933296 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e0f496-fb52-45b0-855a-4eaecf53429f None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:34.933582 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e0f496-fb52-45b0-855a-4eaecf53429f None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:34.933860 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e0f496-fb52-45b0-855a-4eaecf53429f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:34.937490 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b75af1-20b2-40f3-b0e7-d2a78e4a46ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:00:34.945530 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-11b75af1-20b2-40f3-b0e7-d2a78e4a46ba None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 02 19:00:34.945530 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:34.945530 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:34.945530 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:34.945530 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:34.945530 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:34.945530 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:34.945530 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:34.945530 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:34.945530 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:34.945530 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:34.945530 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:34.945530 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:00:34.945530 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:00:34.945530 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:00:34.945530 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:00:34.946422 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:00:34.946422 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:00:34.946422 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:00:34.946422 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:00:34.946422 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:34.946422 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:34.946422 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:34.946422 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:34.946422 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:34.946422 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:34.946422 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:34.946422 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:34.946422 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:34.946422 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:34.946422 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:34.946422 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:34.947246 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:34.947246 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:34.947246 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:34.947246 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:34.947246 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:00:34.947246 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:00:34.947246 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:00:34.947246 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:00:34.947246 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:00:34.947246 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:00:34.947246 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 02 19:00:34.947246 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:00:34.947246 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 257/505] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 2 19:00:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:34.947875 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b75af1-20b2-40f3-b0e7-d2a78e4a46ba None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:35.006142 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e0f496-fb52-45b0-855a-4eaecf53429f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:35.008181 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e0f496-fb52-45b0-855a-4eaecf53429f None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:35.048960 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b75af1-20b2-40f3-b0e7-d2a78e4a46ba None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:35.063052 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:35.082350 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2109b35f-0926-46a0-8ec1-25fe93e8a159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:35.083056 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2109b35f-0926-46a0-8ec1-25fe93e8a159 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:35.083342 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2109b35f-0926-46a0-8ec1-25fe93e8a159 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:35.083623 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2109b35f-0926-46a0-8ec1-25fe93e8a159 None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:35.084351 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2109b35f-0926-46a0-8ec1-25fe93e8a159 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:35.084664 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2109b35f-0926-46a0-8ec1-25fe93e8a159 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:35.086064 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2109b35f-0926-46a0-8ec1-25fe93e8a159 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:35.097990 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 258/506] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:00:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:35.098847 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2109b35f-0926-46a0-8ec1-25fe93e8a159 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:35.111859 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 249/507] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 2 19:00:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 02 19:00:35.111859 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e0f496-fb52-45b0-855a-4eaecf53429f None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:35.159993 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:35.206655 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2109b35f-0926-46a0-8ec1-25fe93e8a159 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:35.208444 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e0f496-fb52-45b0-855a-4eaecf53429f None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:35.240391 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987a08a8-42bb-4c33-9ae9-efa91df8e3b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:35.241062 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987a08a8-42bb-4c33-9ae9-efa91df8e3b5 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:35.241348 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987a08a8-42bb-4c33-9ae9-efa91df8e3b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:35.242190 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987a08a8-42bb-4c33-9ae9-efa91df8e3b5 None admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:35.243063 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987a08a8-42bb-4c33-9ae9-efa91df8e3b5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:35.243378 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987a08a8-42bb-4c33-9ae9-efa91df8e3b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:35.243875 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc45a0bf-73ea-4bc6-8eb4-188293c0f89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:35.244702 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45a0bf-73ea-4bc6-8eb4-188293c0f89a None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:35.244824 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987a08a8-42bb-4c33-9ae9-efa91df8e3b5 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:35.244990 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45a0bf-73ea-4bc6-8eb4-188293c0f89a None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:35.245269 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45a0bf-73ea-4bc6-8eb4-188293c0f89a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:35.255625 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc45a0bf-73ea-4bc6-8eb4-188293c0f89a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:35.257088 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc45a0bf-73ea-4bc6-8eb4-188293c0f89a None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:35.262172 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 259/508] 10.222.0.33 () {56 vars in 1137 bytes} [Thu Jun 2 19:00:35 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:35.263014 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987a08a8-42bb-4c33-9ae9-efa91df8e3b5 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:35.278089 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-fc45a0bf-73ea-4bc6-8eb4-188293c0f89a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:35.278089 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:35.278089 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:35.278089 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:35.278089 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:35.278089 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:35.278089 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:35.278089 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:35.278089 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:35.278089 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:35.278089 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:35.278089 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:35.278089 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:00:35.278089 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:00:35.278089 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:00:35.278975 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:00:35.278975 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:35.278975 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:35.278975 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:35.278975 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:35.278975 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:35.278975 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:35.278975 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:35.278975 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:35.278975 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:35.278975 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:35.278975 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:35.278975 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:35.278975 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:35.278975 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:35.278975 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:35.278975 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:35.279845 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:00:35.279845 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:00:35.279845 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:35.279845 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:00:35.279845 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 250/509] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:00:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:35.280114 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc45a0bf-73ea-4bc6-8eb4-188293c0f89a None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:35.290971 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:35.367084 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987a08a8-42bb-4c33-9ae9-efa91df8e3b5 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:35.378516 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:35.378704 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc45a0bf-73ea-4bc6-8eb4-188293c0f89a None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:35.402922 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a471ca-c9a0-4c1e-8d97-c4a7b0572f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:35.403599 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a471ca-c9a0-4c1e-8d97-c4a7b0572f31 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:35.403888 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a471ca-c9a0-4c1e-8d97-c4a7b0572f31 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:35.404159 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a471ca-c9a0-4c1e-8d97-c4a7b0572f31 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:35.404903 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a471ca-c9a0-4c1e-8d97-c4a7b0572f31 None admin] RBAC: Adding query filter params (name=service) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:35.405212 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a471ca-c9a0-4c1e-8d97-c4a7b0572f31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:35.406584 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a471ca-c9a0-4c1e-8d97-c4a7b0572f31 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:35.414267 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da18649-b1f1-47af-a2db-e12a0f790f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:35.415093 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da18649-b1f1-47af-a2db-e12a0f790f25 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:35.415385 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da18649-b1f1-47af-a2db-e12a0f790f25 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:35.415559 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 260/510] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 2 19:00:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:35.415666 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da18649-b1f1-47af-a2db-e12a0f790f25 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:35.416440 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a471ca-c9a0-4c1e-8d97-c4a7b0572f31 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:35.423403 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da18649-b1f1-47af-a2db-e12a0f790f25 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:35.424849 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da18649-b1f1-47af-a2db-e12a0f790f25 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:35.440534 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-5da18649-b1f1-47af-a2db-e12a0f790f25 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 02 19:00:35.440534 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:35.440534 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:35.440534 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:35.440534 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:35.440534 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:35.440534 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:35.440534 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:35.440534 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:35.440534 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:35.440534 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:35.440534 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:35.440534 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 02 19:00:35.440534 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 02 19:00:35.440534 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 02 19:00:35.440534 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 02 19:00:35.441547 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:35.441547 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:35.441547 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 02 19:00:35.441547 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:00:35.441547 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 02 19:00:35.441547 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:00:35.441547 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:35.441547 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:35.441547 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:35.441547 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:35.441547 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:35.441547 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:35.441547 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:35.441547 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:35.441547 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:35.441547 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:35.441547 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:35.441547 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:35.442424 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:35.442424 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:35.442424 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 02 19:00:35.442424 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 02 19:00:35.442424 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 02 19:00:35.442424 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 02 19:00:35.442424 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 02 19:00:35.442424 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 02 19:00:35.442424 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 02 19:00:35.442424 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:00:35.442424 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 251/511] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 2 19:00:35 2022] GET /identity/v3/users/cinder => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:35.443023 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da18649-b1f1-47af-a2db-e12a0f790f25 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:35.509590 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:35.518842 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a471ca-c9a0-4c1e-8d97-c4a7b0572f31 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:35.541989 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da18649-b1f1-47af-a2db-e12a0f790f25 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:35.551265 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ac8214-15cb-4274-81bd-4e683fb734be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:35.552055 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ac8214-15cb-4274-81bd-4e683fb734be None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:35.552348 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ac8214-15cb-4274-81bd-4e683fb734be None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:35.552623 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ac8214-15cb-4274-81bd-4e683fb734be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:35.563694 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ac8214-15cb-4274-81bd-4e683fb734be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:35.565156 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ac8214-15cb-4274-81bd-4e683fb734be None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:35.576217 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6f92c6-a280-4af3-b582-22112b44844d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:35.576871 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f92c6-a280-4af3-b582-22112b44844d None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:35.577158 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f92c6-a280-4af3-b582-22112b44844d None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:35.577439 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f92c6-a280-4af3-b582-22112b44844d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:35.582663 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-92ac8214-15cb-4274-81bd-4e683fb734be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:35.582663 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:35.582663 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:35.582663 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:35.582663 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:35.582663 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:35.582663 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:35.582663 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:35.582663 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:35.582663 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:35.582663 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:35.582663 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:35.582663 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:00:35.582663 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:00:35.582663 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:00:35.583675 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:00:35.583675 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:35.583675 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:35.583675 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:35.583675 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:35.583675 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:35.583675 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:35.583675 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:35.583675 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:35.583675 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:35.583675 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:35.583675 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:35.583675 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:35.583675 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:35.583675 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:35.583675 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:35.583675 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:35.584548 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:00:35.584548 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:00:35.584548 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:35.584548 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:00:35.584548 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 261/512] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:00:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:35.584814 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ac8214-15cb-4274-81bd-4e683fb734be None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:35.611576 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:35.646334 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6f92c6-a280-4af3-b582-22112b44844d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:35.648536 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6f92c6-a280-4af3-b582-22112b44844d None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:35.685593 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ac8214-15cb-4274-81bd-4e683fb734be None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:35.713095 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:35.718310 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a52d050-efa4-4991-bc21-8a3e867f86b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:35.719061 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a52d050-efa4-4991-bc21-8a3e867f86b6 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:35.719348 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a52d050-efa4-4991-bc21-8a3e867f86b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:35.719625 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a52d050-efa4-4991-bc21-8a3e867f86b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:35.720349 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a52d050-efa4-4991-bc21-8a3e867f86b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:35.720650 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a52d050-efa4-4991-bc21-8a3e867f86b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:35.722030 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a52d050-efa4-4991-bc21-8a3e867f86b6 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:35.734406 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 262/513] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:00:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:35.735227 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a52d050-efa4-4991-bc21-8a3e867f86b6 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:35.753112 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 252/514] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 2 19:00:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 02 19:00:35.753112 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6f92c6-a280-4af3-b582-22112b44844d None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:35.820863 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:35.836330 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a52d050-efa4-4991-bc21-8a3e867f86b6 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:35.857344 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6f92c6-a280-4af3-b582-22112b44844d None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:35.872439 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66281581-df55-46e5-baeb-f93200e32e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:35.873268 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66281581-df55-46e5-baeb-f93200e32e8b None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:35.873560 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66281581-df55-46e5-baeb-f93200e32e8b None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:35.873841 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66281581-df55-46e5-baeb-f93200e32e8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:35.884689 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66281581-df55-46e5-baeb-f93200e32e8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:35.886151 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66281581-df55-46e5-baeb-f93200e32e8b None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:35.892903 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfb3c8c-afe1-4302-9693-179cbddb2f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:35.893577 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb3c8c-afe1-4302-9693-179cbddb2f3f None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:35.893867 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb3c8c-afe1-4302-9693-179cbddb2f3f None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:35.894151 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb3c8c-afe1-4302-9693-179cbddb2f3f None admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:35.895414 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfb3c8c-afe1-4302-9693-179cbddb2f3f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:35.895733 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfb3c8c-afe1-4302-9693-179cbddb2f3f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:35.897149 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfb3c8c-afe1-4302-9693-179cbddb2f3f None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:35.904169 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-66281581-df55-46e5-baeb-f93200e32e8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:35.904169 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:35.904169 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:35.904169 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:35.904169 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:35.904169 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:35.904169 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:35.904169 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:35.904169 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:35.904169 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:35.904169 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:35.904169 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:35.904169 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:00:35.904169 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:00:35.904169 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:00:35.904991 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:00:35.904991 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:35.904991 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:35.904991 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:35.904991 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:35.904991 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:35.904991 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:35.904991 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:35.904991 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:35.904991 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:35.904991 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:35.904991 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:35.904991 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:35.904991 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:35.904991 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:35.904991 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:35.904991 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:35.905822 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:00:35.905822 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:00:35.905822 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:35.905822 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:00:35.905822 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 263/515] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:00:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:35.906493 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66281581-df55-46e5-baeb-f93200e32e8b None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:35.934207 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 253/516] 10.222.0.33 () {56 vars in 1137 bytes} [Thu Jun 2 19:00:35 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:35.935064 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfb3c8c-afe1-4302-9693-179cbddb2f3f None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:35.962430 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:36.005513 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66281581-df55-46e5-baeb-f93200e32e8b None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:36.035730 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfb3c8c-afe1-4302-9693-179cbddb2f3f None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:36.040310 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d115dba9-27e1-4e62-9c1c-e177df168331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:36.040978 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d115dba9-27e1-4e62-9c1c-e177df168331 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:36.041266 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d115dba9-27e1-4e62-9c1c-e177df168331 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:36.041545 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d115dba9-27e1-4e62-9c1c-e177df168331 None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:36.042275 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d115dba9-27e1-4e62-9c1c-e177df168331 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:36.042587 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d115dba9-27e1-4e62-9c1c-e177df168331 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:36.044007 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d115dba9-27e1-4e62-9c1c-e177df168331 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:36.046881 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:36.055672 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 264/517] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:00:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:36.056311 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d115dba9-27e1-4e62-9c1c-e177df168331 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:36.072593 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c171479-62ff-4303-8536-c29921fc6bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:36.072593 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c171479-62ff-4303-8536-c29921fc6bc7 None admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:36.072593 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c171479-62ff-4303-8536-c29921fc6bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:36.072593 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c171479-62ff-4303-8536-c29921fc6bc7 None admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:36.072957 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c171479-62ff-4303-8536-c29921fc6bc7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:36.073231 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c171479-62ff-4303-8536-c29921fc6bc7 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:36.158895 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d115dba9-27e1-4e62-9c1c-e177df168331 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:36.168304 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 254/518] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 2 19:00:35 2022] POST /identity/v3/users => generated 247 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:00:36.169347 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c171479-62ff-4303-8536-c29921fc6bc7 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:36.195019 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6977688-6b26-44de-ac63-cbb008232583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:36.195888 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6977688-6b26-44de-ac63-cbb008232583 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:36.196128 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6977688-6b26-44de-ac63-cbb008232583 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:36.196450 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6977688-6b26-44de-ac63-cbb008232583 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:36.208507 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6977688-6b26-44de-ac63-cbb008232583 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:36.210009 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6977688-6b26-44de-ac63-cbb008232583 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:36.231583 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-e6977688-6b26-44de-ac63-cbb008232583 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:36.231583 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:36.231583 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:36.231583 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:36.231583 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:36.231583 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:36.231583 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:36.231583 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:36.231583 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:36.231583 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:36.231583 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:36.231583 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:36.231583 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:00:36.231583 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:00:36.231583 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:00:36.232541 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:00:36.232541 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:36.232541 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:36.232541 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:36.232541 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:36.232541 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:36.232541 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:36.232541 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:36.232541 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:36.232541 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:36.232541 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:36.232541 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:36.232541 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:36.232541 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:36.232541 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:36.232541 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:36.232541 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:36.238201 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:00:36.238201 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:00:36.238201 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:36.238201 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:00:36.238201 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 265/519] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:00:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:36.238201 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6977688-6b26-44de-ac63-cbb008232583 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:36.293817 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:36.294018 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:36.312534 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c171479-62ff-4303-8536-c29921fc6bc7 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:36.352628 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6977688-6b26-44de-ac63-cbb008232583 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:36.355892 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591dbdb9-f241-445c-8373-2fbf3c0e253a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:36.356831 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591dbdb9-f241-445c-8373-2fbf3c0e253a None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:36.357169 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591dbdb9-f241-445c-8373-2fbf3c0e253a None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:36.357663 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591dbdb9-f241-445c-8373-2fbf3c0e253a None admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:36.358512 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591dbdb9-f241-445c-8373-2fbf3c0e253a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:36.358849 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591dbdb9-f241-445c-8373-2fbf3c0e253a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:36.362650 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591dbdb9-f241-445c-8373-2fbf3c0e253a None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:36.373485 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 255/520] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:00:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:36.374481 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591dbdb9-f241-445c-8373-2fbf3c0e253a None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:36.402278 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a127690d-e594-4499-a4e0-9c252f60ec09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:36.404258 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a127690d-e594-4499-a4e0-9c252f60ec09 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:36.404258 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a127690d-e594-4499-a4e0-9c252f60ec09 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:36.404258 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a127690d-e594-4499-a4e0-9c252f60ec09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:36.493306 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a127690d-e594-4499-a4e0-9c252f60ec09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:36.493652 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591dbdb9-f241-445c-8373-2fbf3c0e253a None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:36.497604 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a127690d-e594-4499-a4e0-9c252f60ec09 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:36.531371 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20882e98-fe55-47d5-a97a-5d1a4f6e8dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:36.532175 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20882e98-fe55-47d5-a97a-5d1a4f6e8dce None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:36.532536 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20882e98-fe55-47d5-a97a-5d1a4f6e8dce None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:36.532826 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20882e98-fe55-47d5-a97a-5d1a4f6e8dce None admin] PATH_INFO: `/v3/projects/service` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:36.540420 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20882e98-fe55-47d5-a97a-5d1a4f6e8dce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:36.541914 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20882e98-fe55-47d5-a97a-5d1a4f6e8dce None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:36.550646 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:36.557924 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-20882e98-fe55-47d5-a97a-5d1a4f6e8dce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:00:36.557924 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:36.557924 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:36.557924 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:36.557924 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:36.557924 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:36.557924 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:36.557924 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:36.557924 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:36.557924 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:36.557924 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:36.557924 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:36.557924 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 02 19:00:36.557924 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 02 19:00:36.557924 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 02 19:00:36.566153 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 02 19:00:36.566153 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:36.566153 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:36.566153 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:36.566153 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:36.566153 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:36.566153 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:36.566153 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:36.566153 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:36.566153 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:36.566153 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:36.566153 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:36.566153 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:36.566153 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:36.566153 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:36.566153 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:36.566153 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:36.567094 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 02 19:00:36.567094 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 02 19:00:36.567094 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 02 19:00:36.567094 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 02 19:00:36.567094 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 02 19:00:36.567094 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 02 19:00:36.567094 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:00:36.567094 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:00:36.567094 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 256/521] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 2 19:00:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:36.567094 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20882e98-fe55-47d5-a97a-5d1a4f6e8dce None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:36.567094 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:36.620757 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 266/522] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 2 19:00:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 02 19:00:36.620757 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a127690d-e594-4499-a4e0-9c252f60ec09 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:36.671972 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20882e98-fe55-47d5-a97a-5d1a4f6e8dce None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:36.708076 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b68d73b-5d97-4492-ab9d-f92269e0579a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:36.708751 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b68d73b-5d97-4492-ab9d-f92269e0579a None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:36.709313 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b68d73b-5d97-4492-ab9d-f92269e0579a None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:36.709313 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b68d73b-5d97-4492-ab9d-f92269e0579a None admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:36.710058 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b68d73b-5d97-4492-ab9d-f92269e0579a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:36.710418 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b68d73b-5d97-4492-ab9d-f92269e0579a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:36.711821 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b68d73b-5d97-4492-ab9d-f92269e0579a None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:36.724637 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 257/523] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:00:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:36.725554 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b68d73b-5d97-4492-ab9d-f92269e0579a None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:36.734128 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a127690d-e594-4499-a4e0-9c252f60ec09 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:36.772042 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8eda81-f9b4-47be-b446-16c3520694ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:36.773391 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8eda81-f9b4-47be-b446-16c3520694ee None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:36.773675 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8eda81-f9b4-47be-b446-16c3520694ee None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:36.773953 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8eda81-f9b4-47be-b446-16c3520694ee None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:36.799956 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8eda81-f9b4-47be-b446-16c3520694ee None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:36.801493 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8eda81-f9b4-47be-b446-16c3520694ee None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:36.818428 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-3d8eda81-f9b4-47be-b446-16c3520694ee None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 02 19:00:36.818428 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:36.818428 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:36.818428 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:36.818428 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:36.818428 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:36.818428 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:36.818428 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:36.818428 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:36.818428 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:36.818428 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:36.818428 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:36.818428 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 02 19:00:36.818428 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 02 19:00:36.818428 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 02 19:00:36.818428 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 02 19:00:36.819752 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:36.819752 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:36.819752 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 02 19:00:36.819752 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:00:36.819752 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 02 19:00:36.819752 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:00:36.819752 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:36.819752 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:36.819752 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:36.819752 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:36.819752 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:36.819752 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:36.819752 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:36.819752 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:36.819752 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:36.819752 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:36.819752 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:36.819752 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:36.820647 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:36.820647 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:36.820647 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 02 19:00:36.820647 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 02 19:00:36.820647 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 02 19:00:36.820647 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 02 19:00:36.820647 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 02 19:00:36.820647 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 02 19:00:36.820647 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 02 19:00:36.820647 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:00:36.820647 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 267/524] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 2 19:00:36 2022] GET /identity/v3/users/neutron => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:36.820647 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8eda81-f9b4-47be-b446-16c3520694ee None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:36.833489 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:36.833714 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:36.874202 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b68d73b-5d97-4492-ab9d-f92269e0579a None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:36.910253 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608e62d8-ba99-42ab-9d66-f8b9555d8c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:36.910960 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608e62d8-ba99-42ab-9d66-f8b9555d8c87 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:36.911251 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608e62d8-ba99-42ab-9d66-f8b9555d8c87 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:36.911533 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608e62d8-ba99-42ab-9d66-f8b9555d8c87 None admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:36.912325 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608e62d8-ba99-42ab-9d66-f8b9555d8c87 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:36.912639 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608e62d8-ba99-42ab-9d66-f8b9555d8c87 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:36.914116 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608e62d8-ba99-42ab-9d66-f8b9555d8c87 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:36.929696 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8eda81-f9b4-47be-b446-16c3520694ee None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:36.938159 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 258/525] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 2 19:00:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:36.938159 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608e62d8-ba99-42ab-9d66-f8b9555d8c87 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:36.968705 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0814a891-b7b0-4c10-a83e-1979d530269f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:36.969507 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0814a891-b7b0-4c10-a83e-1979d530269f None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:36.969790 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0814a891-b7b0-4c10-a83e-1979d530269f None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:36.970074 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0814a891-b7b0-4c10-a83e-1979d530269f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:36.980727 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0814a891-b7b0-4c10-a83e-1979d530269f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:36.982388 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0814a891-b7b0-4c10-a83e-1979d530269f None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:37.000906 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-0814a891-b7b0-4c10-a83e-1979d530269f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:37.000906 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:37.000906 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:37.000906 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:37.000906 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:37.000906 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:37.000906 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:37.000906 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:37.000906 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:37.000906 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:37.000906 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:37.000906 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:37.000906 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:00:37.000906 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:00:37.000906 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:00:37.001841 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:00:37.001841 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:37.001841 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:37.001841 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:37.001841 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:37.001841 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:37.001841 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:37.001841 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:37.001841 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:37.001841 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:37.001841 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:37.001841 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:37.001841 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:37.001841 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:37.001841 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:37.001841 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:37.001841 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:37.002703 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:00:37.002703 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:00:37.002703 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:37.002703 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:00:37.002703 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 268/526] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:00:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:37.003847 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0814a891-b7b0-4c10-a83e-1979d530269f None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:37.046936 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608e62d8-ba99-42ab-9d66-f8b9555d8c87 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:37.072621 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:37.073597 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:37.086147 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027ff423-3775-4278-9995-5f95a2a9787d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:37.086978 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ff423-3775-4278-9995-5f95a2a9787d None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:37.089463 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ff423-3775-4278-9995-5f95a2a9787d None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:37.089463 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ff423-3775-4278-9995-5f95a2a9787d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:37.094540 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027ff423-3775-4278-9995-5f95a2a9787d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:37.096032 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027ff423-3775-4278-9995-5f95a2a9787d None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:37.110628 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-027ff423-3775-4278-9995-5f95a2a9787d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:00:37.110628 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:37.110628 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:37.110628 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:37.110628 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:37.110628 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:37.110628 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:37.110628 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:37.110628 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:37.110628 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:37.110628 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:37.110628 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:37.110628 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 02 19:00:37.110628 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 02 19:00:37.110628 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 02 19:00:37.111536 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 02 19:00:37.111536 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:37.111536 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:37.111536 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:37.111536 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:37.111536 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:37.111536 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:37.111536 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:37.111536 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:37.111536 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:37.111536 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:37.111536 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:37.111536 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:37.111536 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:37.111536 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:37.111536 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:37.111536 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:37.112429 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 02 19:00:37.112429 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 02 19:00:37.112429 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 02 19:00:37.112429 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 02 19:00:37.112429 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 02 19:00:37.112429 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 02 19:00:37.112429 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:00:37.112429 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:00:37.118321 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0814a891-b7b0-4c10-a83e-1979d530269f None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:37.120411 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 259/527] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 2 19:00:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:37.121459 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027ff423-3775-4278-9995-5f95a2a9787d None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:37.161489 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a700ac23-8099-4d49-8b32-34ada4a6ce29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:37.162437 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700ac23-8099-4d49-8b32-34ada4a6ce29 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:37.162959 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700ac23-8099-4d49-8b32-34ada4a6ce29 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:37.164032 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700ac23-8099-4d49-8b32-34ada4a6ce29 None admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:37.165602 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a700ac23-8099-4d49-8b32-34ada4a6ce29 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:37.166097 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a700ac23-8099-4d49-8b32-34ada4a6ce29 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:37.168637 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a700ac23-8099-4d49-8b32-34ada4a6ce29 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:37.192105 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 269/528] 10.222.0.33 () {56 vars in 1139 bytes} [Thu Jun 2 19:00:37 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:37.194068 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a700ac23-8099-4d49-8b32-34ada4a6ce29 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:37.229246 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027ff423-3775-4278-9995-5f95a2a9787d None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:37.264099 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a3d47e-a202-45a8-86ba-9bbf6e981281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:37.265047 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a3d47e-a202-45a8-86ba-9bbf6e981281 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:37.265345 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a3d47e-a202-45a8-86ba-9bbf6e981281 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:37.265607 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a3d47e-a202-45a8-86ba-9bbf6e981281 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:37.266443 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a3d47e-a202-45a8-86ba-9bbf6e981281 None admin] RBAC: Adding query filter params (role.id=0e56fba112f94ebba363ffab7f4aa9fe, scope.project.id=8c04020eaebd4647804cf8653f32d9e5, user.id=70664d5cbdee47ee89c120761939cc75) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:37.266771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a3d47e-a202-45a8-86ba-9bbf6e981281 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:37.268211 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a3d47e-a202-45a8-86ba-9bbf6e981281 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:37.275028 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 260/529] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 2 19:00:37 2022] GET /identity/v3/role_assignments?user.id=70664d5cbdee47ee89c120761939cc75&scope.project.id=8c04020eaebd4647804cf8653f32d9e5&role.id=0e56fba112f94ebba363ffab7f4aa9fe => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:37.276025 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a3d47e-a202-45a8-86ba-9bbf6e981281 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:37.320689 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a700ac23-8099-4d49-8b32-34ada4a6ce29 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:37.338686 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:37.341554 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:37.359886 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926b7f69-f216-4bf1-8193-f27bf46e56ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:37.369959 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b7f69-f216-4bf1-8193-f27bf46e56ff None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:37.369959 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b7f69-f216-4bf1-8193-f27bf46e56ff None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:37.369959 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b7f69-f216-4bf1-8193-f27bf46e56ff None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:37.369959 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926b7f69-f216-4bf1-8193-f27bf46e56ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:37.369959 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926b7f69-f216-4bf1-8193-f27bf46e56ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:37.369959 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926b7f69-f216-4bf1-8193-f27bf46e56ff None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:37.380834 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 270/530] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:00:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:37.382590 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926b7f69-f216-4bf1-8193-f27bf46e56ff None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:37.407952 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a3d47e-a202-45a8-86ba-9bbf6e981281 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:37.460888 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9972ec3e-b662-4a19-9978-4998d580dbed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:37.461507 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9972ec3e-b662-4a19-9978-4998d580dbed None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:37.462076 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9972ec3e-b662-4a19-9978-4998d580dbed None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:37.462363 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9972ec3e-b662-4a19-9978-4998d580dbed None admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:37.463222 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9972ec3e-b662-4a19-9978-4998d580dbed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:37.463541 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9972ec3e-b662-4a19-9978-4998d580dbed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:37.465064 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9972ec3e-b662-4a19-9978-4998d580dbed None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:37.488555 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 261/531] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 2 19:00:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:37.489907 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9972ec3e-b662-4a19-9978-4998d580dbed None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:37.521412 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926b7f69-f216-4bf1-8193-f27bf46e56ff None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:37.565621 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f260a5e8-f655-4688-91f5-1035371ba020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:37.566909 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f260a5e8-f655-4688-91f5-1035371ba020 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:37.567208 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f260a5e8-f655-4688-91f5-1035371ba020 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:37.567489 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f260a5e8-f655-4688-91f5-1035371ba020 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:37.607159 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9972ec3e-b662-4a19-9978-4998d580dbed None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:37.626382 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:37.641386 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:37.645054 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c039fc26-323c-494a-bc73-ab2cb14e3ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:37.646075 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c039fc26-323c-494a-bc73-ab2cb14e3ab5 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:37.646576 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c039fc26-323c-494a-bc73-ab2cb14e3ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:37.647095 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c039fc26-323c-494a-bc73-ab2cb14e3ab5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:37.655866 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f260a5e8-f655-4688-91f5-1035371ba020 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:37.657616 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f260a5e8-f655-4688-91f5-1035371ba020 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:37.658531 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c039fc26-323c-494a-bc73-ab2cb14e3ab5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:37.662012 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c039fc26-323c-494a-bc73-ab2cb14e3ab5 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:37.683209 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-c039fc26-323c-494a-bc73-ab2cb14e3ab5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:00:37.683209 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:37.683209 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:37.683209 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:37.683209 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:37.683209 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:37.683209 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:37.683209 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:37.683209 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:37.683209 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:37.683209 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:37.683209 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:37.683209 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 02 19:00:37.683209 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 02 19:00:37.683209 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 02 19:00:37.684108 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 02 19:00:37.684108 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:37.684108 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:37.684108 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:37.684108 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:37.684108 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:37.684108 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:37.684108 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:37.684108 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:37.684108 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:37.684108 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:37.684108 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:37.684108 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:37.684108 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:37.684108 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:37.684108 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:37.684108 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:37.685345 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 02 19:00:37.685345 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 02 19:00:37.685345 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 02 19:00:37.685345 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 02 19:00:37.685345 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 02 19:00:37.685345 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 02 19:00:37.685345 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:00:37.685345 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:00:37.685345 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 262/532] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 2 19:00:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:37.687390 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c039fc26-323c-494a-bc73-ab2cb14e3ab5 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:37.782371 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 271/533] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 2 19:00:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 02 19:00:37.783223 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f260a5e8-f655-4688-91f5-1035371ba020 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:37.794151 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c039fc26-323c-494a-bc73-ab2cb14e3ab5 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:37.829074 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9f60d6-966c-4311-b893-26db7f9541ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:37.830258 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f60d6-966c-4311-b893-26db7f9541ed None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:37.830743 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f60d6-966c-4311-b893-26db7f9541ed None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:37.831263 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f60d6-966c-4311-b893-26db7f9541ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:37.832106 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9f60d6-966c-4311-b893-26db7f9541ed None admin] RBAC: Adding query filter params (role.id=0e56fba112f94ebba363ffab7f4aa9fe, scope.project.id=8c04020eaebd4647804cf8653f32d9e5, user.id=e07892c0aa21449b94a1cc78e3b8a653) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:37.832470 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9f60d6-966c-4311-b893-26db7f9541ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:37.833960 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9f60d6-966c-4311-b893-26db7f9541ed None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:37.840593 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 263/534] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 2 19:00:37 2022] GET /identity/v3/role_assignments?user.id=e07892c0aa21449b94a1cc78e3b8a653&scope.project.id=8c04020eaebd4647804cf8653f32d9e5&role.id=0e56fba112f94ebba363ffab7f4aa9fe => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:37.841434 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9f60d6-966c-4311-b893-26db7f9541ed None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:37.879241 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:37.891613 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:37.920331 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f260a5e8-f655-4688-91f5-1035371ba020 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:37.956047 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9f60d6-966c-4311-b893-26db7f9541ed None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:37.962637 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a8c1d7-e609-4941-9d1e-d75eb3fb5fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:37.963624 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a8c1d7-e609-4941-9d1e-d75eb3fb5fb9 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:37.964139 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a8c1d7-e609-4941-9d1e-d75eb3fb5fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:37.964685 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a8c1d7-e609-4941-9d1e-d75eb3fb5fb9 None admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:37.965776 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a8c1d7-e609-4941-9d1e-d75eb3fb5fb9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:37.966349 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a8c1d7-e609-4941-9d1e-d75eb3fb5fb9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:37.968271 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a8c1d7-e609-4941-9d1e-d75eb3fb5fb9 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:37.989526 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 272/535] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 2 19:00:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:37.996144 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a8c1d7-e609-4941-9d1e-d75eb3fb5fb9 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:37.998035 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f0f84b-3e43-4454-ace6-b2fd7515da17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:37.998889 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f0f84b-3e43-4454-ace6-b2fd7515da17 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:37.999226 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f0f84b-3e43-4454-ace6-b2fd7515da17 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:37.999508 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f0f84b-3e43-4454-ace6-b2fd7515da17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:38.014781 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f0f84b-3e43-4454-ace6-b2fd7515da17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:38.014781 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f0f84b-3e43-4454-ace6-b2fd7515da17 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:38.033644 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-96f0f84b-3e43-4454-ace6-b2fd7515da17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:38.033644 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:38.033644 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:38.033644 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:38.033644 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:38.033644 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:38.033644 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:38.033644 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:38.033644 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:38.033644 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:38.033644 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:38.033644 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:38.033644 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:00:38.033644 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:00:38.033644 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:00:38.034517 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:00:38.034517 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:38.034517 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:38.034517 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:38.034517 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:38.034517 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:38.034517 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:38.034517 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:38.034517 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:38.034517 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:38.034517 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:38.034517 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:38.034517 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:38.034517 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:38.034517 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:38.034517 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:38.034517 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:38.035418 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:00:38.035418 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:00:38.035418 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:38.035418 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:00:38.035418 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 264/536] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:00:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:38.048058 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f0f84b-3e43-4454-ace6-b2fd7515da17 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:38.163269 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:38.195848 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f0f84b-3e43-4454-ace6-b2fd7515da17 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:38.220364 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a8c1d7-e609-4941-9d1e-d75eb3fb5fb9 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:38.260788 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d2f361-8afe-436f-9517-843150e5572b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:38.261460 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d2f361-8afe-436f-9517-843150e5572b None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:38.261742 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d2f361-8afe-436f-9517-843150e5572b None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:38.262016 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d2f361-8afe-436f-9517-843150e5572b None admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:38.262767 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d2f361-8afe-436f-9517-843150e5572b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:38.263081 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d2f361-8afe-436f-9517-843150e5572b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:38.264464 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d2f361-8afe-436f-9517-843150e5572b None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:38.289080 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 265/537] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:00:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:38.295518 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4a2071-b605-4cae-a767-ec2410234d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:38.296209 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a2071-b605-4cae-a767-ec2410234d0c None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:38.296494 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a2071-b605-4cae-a767-ec2410234d0c None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:38.296830 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a2071-b605-4cae-a767-ec2410234d0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:38.297694 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4a2071-b605-4cae-a767-ec2410234d0c None admin] RBAC: Adding query filter params (role.id=0e56fba112f94ebba363ffab7f4aa9fe, scope.project.id=8c04020eaebd4647804cf8653f32d9e5, user.id=091860b2577f49c5af61c75803a90c0d) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:38.298009 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4a2071-b605-4cae-a767-ec2410234d0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:38.299509 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4a2071-b605-4cae-a767-ec2410234d0c None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:38.299748 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d2f361-8afe-436f-9517-843150e5572b None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:38.308001 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 273/538] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 2 19:00:37 2022] GET /identity/v3/role_assignments?user.id=091860b2577f49c5af61c75803a90c0d&scope.project.id=8c04020eaebd4647804cf8653f32d9e5&role.id=0e56fba112f94ebba363ffab7f4aa9fe => generated 259 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:00:38.459748 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d2f361-8afe-436f-9517-843150e5572b None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:38.497257 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57092d6f-65e9-47c0-aa3a-f278883980b1 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:38.497659 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57092d6f-65e9-47c0-aa3a-f278883980b1 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:38.497938 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57092d6f-65e9-47c0-aa3a-f278883980b1 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:38.499568 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 274/539] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:00:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 02 19:00:38.505583 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dceca5d-b029-4614-adf6-aee1d05b02fd None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:38.505920 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dceca5d-b029-4614-adf6-aee1d05b02fd None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:38.506231 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dceca5d-b029-4614-adf6-aee1d05b02fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:38.514160 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:38.527086 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c02c725-dfe7-4d57-aa6d-509a83ac763a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:38.527915 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c02c725-dfe7-4d57-aa6d-509a83ac763a None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:38.528219 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c02c725-dfe7-4d57-aa6d-509a83ac763a None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:38.531921 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c02c725-dfe7-4d57-aa6d-509a83ac763a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:38.534472 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-4dceca5d-b029-4614-adf6-aee1d05b02fd None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:00:38.536029 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-4dceca5d-b029-4614-adf6-aee1d05b02fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:00:38.548493 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c02c725-dfe7-4d57-aa6d-509a83ac763a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:38.550312 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c02c725-dfe7-4d57-aa6d-509a83ac763a None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:38.571434 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-8c02c725-dfe7-4d57-aa6d-509a83ac763a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:00:38.571434 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:38.571434 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:38.571434 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:38.571434 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:38.571434 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:38.571434 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:38.571434 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:38.571434 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:38.571434 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:38.571434 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:38.571434 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:38.571434 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 02 19:00:38.571434 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 02 19:00:38.571434 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 02 19:00:38.572332 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 02 19:00:38.572332 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:38.572332 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:38.572332 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:38.572332 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:38.572332 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:38.572332 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:38.572332 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:38.572332 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:38.572332 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:38.572332 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:38.572332 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:38.572332 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:38.572332 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:38.572332 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:38.572332 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:38.572332 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:38.573154 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 02 19:00:38.573154 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 02 19:00:38.573154 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 02 19:00:38.573154 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 02 19:00:38.573154 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 02 19:00:38.573154 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 02 19:00:38.573154 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:00:38.573154 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:00:38.575369 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 266/540] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 2 19:00:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:38.581155 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c02c725-dfe7-4d57-aa6d-509a83ac763a None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:38.584528 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:38.646505 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 275/541] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:00:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 19:00:38.678303 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dceca5d-b029-4614-adf6-aee1d05b02fd None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:38.793871 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c02c725-dfe7-4d57-aa6d-509a83ac763a None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:38.862258 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dceca5d-b029-4614-adf6-aee1d05b02fd None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:38.870179 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8045b993-b355-42b6-829e-04b613e522ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:38.871300 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8045b993-b355-42b6-829e-04b613e522ae None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:38.873219 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8045b993-b355-42b6-829e-04b613e522ae None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:38.877032 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8045b993-b355-42b6-829e-04b613e522ae None admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:38.878507 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8045b993-b355-42b6-829e-04b613e522ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:38.879093 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8045b993-b355-42b6-829e-04b613e522ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:38.881011 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8045b993-b355-42b6-829e-04b613e522ae None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:38.895445 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 267/542] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 2 19:00:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:00:38.907857 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8045b993-b355-42b6-829e-04b613e522ae None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:38.912016 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:38.916954 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f53b24-dca5-48d8-b453-b50a528a0ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:38.917886 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f53b24-dca5-48d8-b453-b50a528a0ff3 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:38.918330 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f53b24-dca5-48d8-b453-b50a528a0ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:38.918707 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f53b24-dca5-48d8-b453-b50a528a0ff3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:38.931706 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f53b24-dca5-48d8-b453-b50a528a0ff3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:00:38.940158 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-92f53b24-dca5-48d8-b453-b50a528a0ff3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 02 19:00:38.940158 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:38.940158 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:38.940158 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:38.940158 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:38.940158 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:38.940158 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:38.940158 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:38.940158 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:38.940158 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:38.940158 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:38.940158 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:38.940158 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:00:38.940158 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:00:38.940158 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:00:38.940158 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:00:38.941232 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:00:38.941232 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:00:38.941232 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:00:38.941232 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:00:38.941232 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:38.941232 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:38.941232 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:38.941232 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:38.941232 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:38.941232 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:38.941232 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:38.941232 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:38.941232 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:38.941232 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:38.941232 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:38.941232 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:38.942090 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:38.942090 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:38.942090 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:38.942090 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:38.942090 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:00:38.942090 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:00:38.942090 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:00:38.942090 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:00:38.942090 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:00:38.942090 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:00:38.942090 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 02 19:00:38.942090 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:00:38.943039 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 276/543] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 2 19:00:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 02 19:00:38.948839 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f53b24-dca5-48d8-b453-b50a528a0ff3 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:39.037238 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:39.077547 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8045b993-b355-42b6-829e-04b613e522ae None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:39.140851 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2b2e5b-0cb8-4148-af18-8c338697bac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:39.142444 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b2e5b-0cb8-4148-af18-8c338697bac6 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:39.143109 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b2e5b-0cb8-4148-af18-8c338697bac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:39.144582 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b2e5b-0cb8-4148-af18-8c338697bac6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:39.145677 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2b2e5b-0cb8-4148-af18-8c338697bac6 None admin] RBAC: Adding query filter params (role.id=0e56fba112f94ebba363ffab7f4aa9fe, scope.project.id=8c04020eaebd4647804cf8653f32d9e5, user.id=7417a54046e34e899d7b03f2f2314deb) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:39.146879 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2b2e5b-0cb8-4148-af18-8c338697bac6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:39.149027 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2b2e5b-0cb8-4148-af18-8c338697bac6 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:39.159143 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f53b24-dca5-48d8-b453-b50a528a0ff3 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:39.162147 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 268/544] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 2 19:00:38 2022] GET /identity/v3/role_assignments?user.id=7417a54046e34e899d7b03f2f2314deb&scope.project.id=8c04020eaebd4647804cf8653f32d9e5&role.id=0e56fba112f94ebba363ffab7f4aa9fe => generated 259 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:39.213876 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6f1440-289d-406a-a405-efca154c6571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:39.214953 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6f1440-289d-406a-a405-efca154c6571 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:39.215441 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6f1440-289d-406a-a405-efca154c6571 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:39.215956 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6f1440-289d-406a-a405-efca154c6571 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:39.216928 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6f1440-289d-406a-a405-efca154c6571 None admin] RBAC: Adding query filter params (name=service) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:39.217549 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6f1440-289d-406a-a405-efca154c6571 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:39.219275 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6f1440-289d-406a-a405-efca154c6571 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:39.229652 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 277/545] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 2 19:00:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:39.240165 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2b2e5b-0cb8-4148-af18-8c338697bac6 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:39.395347 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2b2e5b-0cb8-4148-af18-8c338697bac6 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:39.433446 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:39.442564 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618172bc-053a-4bfe-a48a-f8d6194897da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:39.443673 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618172bc-053a-4bfe-a48a-f8d6194897da None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:39.444124 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618172bc-053a-4bfe-a48a-f8d6194897da None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:39.444561 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618172bc-053a-4bfe-a48a-f8d6194897da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:39.460361 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618172bc-053a-4bfe-a48a-f8d6194897da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:39.462604 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618172bc-053a-4bfe-a48a-f8d6194897da None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:39.483192 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-618172bc-053a-4bfe-a48a-f8d6194897da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:39.483192 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:39.483192 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:39.483192 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:39.483192 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:39.483192 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:39.483192 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:39.483192 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:39.483192 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:39.483192 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:39.483192 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:39.483192 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:39.483192 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:00:39.483192 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:00:39.483192 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:00:39.484076 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:00:39.484076 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:39.484076 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:39.484076 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:39.484076 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:39.484076 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:39.484076 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:39.484076 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:39.484076 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:39.484076 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:39.484076 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:39.484076 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:39.484076 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:39.484076 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:39.484076 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:39.484076 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:39.484076 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:39.485320 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:00:39.485320 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:00:39.485320 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:39.485320 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:00:39.491301 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 269/546] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:00:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:39.492686 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6f1440-289d-406a-a405-efca154c6571 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:39.589405 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:39.652791 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6f1440-289d-406a-a405-efca154c6571 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:39.705038 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb24932e-3c27-43b3-b866-417f44751031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:39.705840 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb24932e-3c27-43b3-b866-417f44751031 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:39.706127 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb24932e-3c27-43b3-b866-417f44751031 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:39.706406 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb24932e-3c27-43b3-b866-417f44751031 None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:39.707215 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb24932e-3c27-43b3-b866-417f44751031 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:39.707527 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb24932e-3c27-43b3-b866-417f44751031 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:39.709010 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb24932e-3c27-43b3-b866-417f44751031 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:39.724960 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 278/547] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:00:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:00:39.730914 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618172bc-053a-4bfe-a48a-f8d6194897da None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:39.871969 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618172bc-053a-4bfe-a48a-f8d6194897da None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:39.913242 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c465545-c02e-40c8-b04a-acfaeec86cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:39.915855 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c465545-c02e-40c8-b04a-acfaeec86cb3 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:39.915855 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c465545-c02e-40c8-b04a-acfaeec86cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:39.915855 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c465545-c02e-40c8-b04a-acfaeec86cb3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:39.923833 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c465545-c02e-40c8-b04a-acfaeec86cb3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:39.925308 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c465545-c02e-40c8-b04a-acfaeec86cb3 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:39.941068 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-2c465545-c02e-40c8-b04a-acfaeec86cb3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 02 19:00:39.941068 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:39.941068 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:39.941068 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:39.941068 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:39.941068 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:39.941068 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:39.941068 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:39.941068 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:39.941068 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:39.941068 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:39.941068 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:39.941068 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 02 19:00:39.941068 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 02 19:00:39.941068 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 02 19:00:39.941068 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 02 19:00:39.942029 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:39.942029 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:39.942029 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 02 19:00:39.942029 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:00:39.942029 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 02 19:00:39.942029 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:00:39.942029 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:39.942029 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:39.942029 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:39.942029 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:39.942029 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:39.942029 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:39.942029 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:39.942029 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:39.942029 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:39.942029 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:39.942029 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:39.942029 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:39.943394 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:39.943394 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:39.943394 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 02 19:00:39.943394 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 02 19:00:39.943394 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 02 19:00:39.943394 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 02 19:00:39.943394 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 02 19:00:39.943394 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 02 19:00:39.943394 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 02 19:00:39.943394 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:00:39.943394 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 270/548] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 2 19:00:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:39.949318 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb24932e-3c27-43b3-b866-417f44751031 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:40.095192 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb24932e-3c27-43b3-b866-417f44751031 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:40.117299 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:40.144443 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15efead-c9e0-4f67-b2e8-0180832b9896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:40.145137 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15efead-c9e0-4f67-b2e8-0180832b9896 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:40.145421 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15efead-c9e0-4f67-b2e8-0180832b9896 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:40.145698 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15efead-c9e0-4f67-b2e8-0180832b9896 None admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:40.146549 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15efead-c9e0-4f67-b2e8-0180832b9896 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:40.146884 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15efead-c9e0-4f67-b2e8-0180832b9896 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:40.148479 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15efead-c9e0-4f67-b2e8-0180832b9896 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:40.174188 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 279/549] 10.222.0.33 () {56 vars in 1135 bytes} [Thu Jun 2 19:00:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:40.183895 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c465545-c02e-40c8-b04a-acfaeec86cb3 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:40.219902 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af51369-783d-4c87-9ee8-c55873a627a6 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:40.220246 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af51369-783d-4c87-9ee8-c55873a627a6 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:40.220465 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af51369-783d-4c87-9ee8-c55873a627a6 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:40.222118 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 280/550] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:00:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:00:40.232561 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66536a61-d641-4801-8450-c22c97fd1b7f None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:40.233597 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66536a61-d641-4801-8450-c22c97fd1b7f None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:40.234073 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66536a61-d641-4801-8450-c22c97fd1b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:40.247181 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:40.267800 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-66536a61-d641-4801-8450-c22c97fd1b7f None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:00:40.269008 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-66536a61-d641-4801-8450-c22c97fd1b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:00:40.313356 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c465545-c02e-40c8-b04a-acfaeec86cb3 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:40.352836 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fb9a38-82f9-49ed-8f36-2c5bffb743f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:40.354034 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb9a38-82f9-49ed-8f36-2c5bffb743f0 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:40.354327 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb9a38-82f9-49ed-8f36-2c5bffb743f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:40.354605 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb9a38-82f9-49ed-8f36-2c5bffb743f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:40.436269 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 281/551] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:00:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 19:00:40.452908 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fb9a38-82f9-49ed-8f36-2c5bffb743f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:40.454653 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fb9a38-82f9-49ed-8f36-2c5bffb743f0 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:40.486900 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66536a61-d641-4801-8450-c22c97fd1b7f None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:40.523860 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:40.619721 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 271/552] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 2 19:00:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 02 19:00:40.621688 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fb9a38-82f9-49ed-8f36-2c5bffb743f0 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:40.687137 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66536a61-d641-4801-8450-c22c97fd1b7f None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:40.689280 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:40.728746 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba5a9e-2f45-4d39-b6fa-27ba779af845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:40.729940 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba5a9e-2f45-4d39-b6fa-27ba779af845 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:40.730405 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba5a9e-2f45-4d39-b6fa-27ba779af845 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:40.730929 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba5a9e-2f45-4d39-b6fa-27ba779af845 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:40.743553 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ba5a9e-2f45-4d39-b6fa-27ba779af845 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:00:40.752402 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-e1ba5a9e-2f45-4d39-b6fa-27ba779af845 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 02 19:00:40.752402 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:40.752402 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:40.752402 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:40.752402 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:40.752402 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:40.752402 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:40.752402 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:40.752402 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:40.752402 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:40.752402 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:40.752402 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:40.752402 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:00:40.752402 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:00:40.752402 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:00:40.752402 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:00:40.753750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:00:40.753750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:00:40.753750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:00:40.753750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:00:40.753750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:40.753750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:40.753750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:40.753750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:40.753750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:40.753750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:40.753750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:40.753750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:40.753750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:40.753750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:40.753750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:40.753750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:40.754560 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:40.754560 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:40.754560 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:40.754560 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:40.754560 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:00:40.754560 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:00:40.754560 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:00:40.754560 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:00:40.754560 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:00:40.754560 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:00:40.754560 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 02 19:00:40.754560 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:00:40.757523 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 282/553] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 2 19:00:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 02 19:00:40.764570 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba5a9e-2f45-4d39-b6fa-27ba779af845 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:40.801956 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fb9a38-82f9-49ed-8f36-2c5bffb743f0 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:40.896154 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d87e56-a2ae-4607-9b36-94af9bd9f203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:40.898502 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d87e56-a2ae-4607-9b36-94af9bd9f203 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:40.898980 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d87e56-a2ae-4607-9b36-94af9bd9f203 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:40.899421 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d87e56-a2ae-4607-9b36-94af9bd9f203 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:40.910259 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:40.917607 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d87e56-a2ae-4607-9b36-94af9bd9f203 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:40.920403 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d87e56-a2ae-4607-9b36-94af9bd9f203 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:40.948172 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-87d87e56-a2ae-4607-9b36-94af9bd9f203 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:40.948172 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:40.948172 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:40.948172 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:40.948172 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:40.948172 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:40.948172 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:40.948172 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:40.948172 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:40.948172 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:40.948172 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:40.948172 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:40.948172 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:00:40.948172 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:00:40.948172 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:00:40.949236 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:00:40.949236 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:40.949236 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:40.949236 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:40.949236 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:40.949236 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:40.949236 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:40.949236 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:40.949236 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:40.949236 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:40.949236 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:40.949236 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:40.949236 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:40.949236 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:40.949236 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:40.949236 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:40.949236 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:40.950376 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:00:40.950376 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:00:40.950376 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:40.950376 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:00:40.957191 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 272/554] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:00:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 02 19:00:40.957191 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8011ae-ae30-4d7b-b975-58018f858d5e None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:40.957191 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8011ae-ae30-4d7b-b975-58018f858d5e None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:40.957191 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8011ae-ae30-4d7b-b975-58018f858d5e None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:40.957191 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 273/555] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:00:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:00:40.962158 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8011ae-ae30-4d7b-b975-58018f858d5e None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:40.965580 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba5a9e-2f45-4d39-b6fa-27ba779af845 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:41.006698 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f952a9f0-e071-4f13-9e8a-9238d1ec99b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:41.007791 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f952a9f0-e071-4f13-9e8a-9238d1ec99b5 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:41.008257 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f952a9f0-e071-4f13-9e8a-9238d1ec99b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:41.008729 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f952a9f0-e071-4f13-9e8a-9238d1ec99b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:41.009691 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f952a9f0-e071-4f13-9e8a-9238d1ec99b5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:41.010208 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f952a9f0-e071-4f13-9e8a-9238d1ec99b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:41.012010 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f952a9f0-e071-4f13-9e8a-9238d1ec99b5 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:41.022163 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 283/556] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 2 19:00:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:41.024170 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b22b783-5fb1-43ee-84c0-765dae0074c8 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:41.024663 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b22b783-5fb1-43ee-84c0-765dae0074c8 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:41.025156 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b22b783-5fb1-43ee-84c0-765dae0074c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:41.046360 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-0b22b783-5fb1-43ee-84c0-765dae0074c8 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:00:41.048326 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-0b22b783-5fb1-43ee-84c0-765dae0074c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:00:41.070658 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:41.086231 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8011ae-ae30-4d7b-b975-58018f858d5e None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:41.125097 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afef61dc-0449-4902-94b1-f19cc5493f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:41.126123 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afef61dc-0449-4902-94b1-f19cc5493f75 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:41.126625 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afef61dc-0449-4902-94b1-f19cc5493f75 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:41.127126 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afef61dc-0449-4902-94b1-f19cc5493f75 None admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:41.128209 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afef61dc-0449-4902-94b1-f19cc5493f75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:41.128973 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afef61dc-0449-4902-94b1-f19cc5493f75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:41.130818 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afef61dc-0449-4902-94b1-f19cc5493f75 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:41.143601 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 284/557] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:00:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 19:00:41.144830 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b22b783-5fb1-43ee-84c0-765dae0074c8 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:41.154592 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 274/558] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:00:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:41.165002 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afef61dc-0449-4902-94b1-f19cc5493f75 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:41.222600 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:41.260889 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b22b783-5fb1-43ee-84c0-765dae0074c8 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:41.279622 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afef61dc-0449-4902-94b1-f19cc5493f75 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:41.299078 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5719948-1ee7-45c3-8bde-7d211e29df8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:41.299928 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5719948-1ee7-45c3-8bde-7d211e29df8b None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:41.300214 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5719948-1ee7-45c3-8bde-7d211e29df8b None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:41.300496 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5719948-1ee7-45c3-8bde-7d211e29df8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:41.310969 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5719948-1ee7-45c3-8bde-7d211e29df8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:41.312487 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5719948-1ee7-45c3-8bde-7d211e29df8b None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:41.316365 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5ebdcb-c19b-4f04-8de7-281d3b9524dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:41.317168 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5ebdcb-c19b-4f04-8de7-281d3b9524dd None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:41.317451 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5ebdcb-c19b-4f04-8de7-281d3b9524dd None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:41.317725 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5ebdcb-c19b-4f04-8de7-281d3b9524dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:41.324798 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5ebdcb-c19b-4f04-8de7-281d3b9524dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:41.326235 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5ebdcb-c19b-4f04-8de7-281d3b9524dd None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:41.330771 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-e5719948-1ee7-45c3-8bde-7d211e29df8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:41.330771 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:41.330771 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:41.330771 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:41.330771 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:41.330771 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:41.330771 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:41.330771 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:41.330771 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:41.330771 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:41.330771 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:41.330771 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:41.330771 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:00:41.330771 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:00:41.330771 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:00:41.331649 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:00:41.331649 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:41.331649 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:41.331649 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:41.331649 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:41.331649 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:41.331649 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:41.331649 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:41.331649 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:41.331649 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:41.331649 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:41.331649 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:41.331649 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:41.331649 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:41.331649 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:41.331649 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:41.331649 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:41.332518 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:00:41.332518 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:00:41.332518 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:41.332518 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:00:41.332518 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 285/559] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:00:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:41.333410 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91715b3-c80e-49ea-b658-ece2bfedbc95 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:41.333701 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91715b3-c80e-49ea-b658-ece2bfedbc95 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:41.333980 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91715b3-c80e-49ea-b658-ece2bfedbc95 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:41.339142 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 286/560] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:00:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:00:41.339142 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91715b3-c80e-49ea-b658-ece2bfedbc95 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:41.345254 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-2e5ebdcb-c19b-4f04-8de7-281d3b9524dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:00:41.345254 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:41.345254 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:41.345254 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:41.345254 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:41.345254 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:41.345254 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:41.345254 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:41.345254 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:41.345254 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:41.345254 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:41.345254 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:41.345254 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 02 19:00:41.345254 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 02 19:00:41.345254 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 02 19:00:41.346318 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 02 19:00:41.346318 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:41.346318 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:41.346318 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:41.346318 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:41.346318 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:41.346318 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:41.346318 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:41.346318 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:41.346318 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:41.346318 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:41.346318 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:41.346318 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:41.346318 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:41.346318 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:41.346318 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:41.346318 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:41.352435 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 02 19:00:41.352435 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 02 19:00:41.352435 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 02 19:00:41.352435 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 02 19:00:41.352435 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 02 19:00:41.352435 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 02 19:00:41.352435 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:00:41.352435 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:00:41.352435 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 275/561] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 2 19:00:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 02 19:00:41.352435 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5ebdcb-c19b-4f04-8de7-281d3b9524dd None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:41.355231 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:41.464023 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91715b3-c80e-49ea-b658-ece2bfedbc95 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:41.471308 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5ebdcb-c19b-4f04-8de7-281d3b9524dd None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:41.486105 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:41.502263 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f705489-955c-427a-b441-e0120d596b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:41.503104 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f705489-955c-427a-b441-e0120d596b67 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:41.503649 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f705489-955c-427a-b441-e0120d596b67 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:41.504109 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f705489-955c-427a-b441-e0120d596b67 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:41.512922 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af01677f-6fc4-4dff-9fb1-2102d4bbd560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:41.513606 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af01677f-6fc4-4dff-9fb1-2102d4bbd560 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:41.513892 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af01677f-6fc4-4dff-9fb1-2102d4bbd560 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:41.514172 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af01677f-6fc4-4dff-9fb1-2102d4bbd560 None admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:41.514938 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af01677f-6fc4-4dff-9fb1-2102d4bbd560 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:41.515275 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af01677f-6fc4-4dff-9fb1-2102d4bbd560 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:41.516652 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af01677f-6fc4-4dff-9fb1-2102d4bbd560 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:41.517522 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f705489-955c-427a-b441-e0120d596b67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:00:41.527593 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-1f705489-955c-427a-b441-e0120d596b67 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 02 19:00:41.527593 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:41.527593 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:41.527593 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:41.527593 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:41.527593 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:41.527593 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:41.527593 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:41.527593 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:41.527593 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:41.527593 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:41.527593 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:41.527593 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:00:41.527593 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:00:41.527593 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:00:41.527593 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:00:41.528560 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:00:41.528560 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:00:41.528560 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:00:41.528560 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:00:41.528560 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:41.528560 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:41.528560 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:41.528560 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:41.528560 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:41.528560 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:41.528560 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:41.528560 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:41.528560 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:41.528560 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:41.528560 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:41.528560 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:41.529447 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:41.529447 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:41.529447 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:41.529447 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:41.529447 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:00:41.529447 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:00:41.529447 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:00:41.529447 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:00:41.529447 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:00:41.529447 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:00:41.529447 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 02 19:00:41.529447 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:00:41.535772 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 287/562] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 2 19:00:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:41.535772 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f638479-e54c-4de9-b2cd-929a5a3cf5c7 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:41.535772 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f638479-e54c-4de9-b2cd-929a5a3cf5c7 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:41.535772 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f638479-e54c-4de9-b2cd-929a5a3cf5c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:41.536000 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 276/563] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:00:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:41.536000 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af01677f-6fc4-4dff-9fb1-2102d4bbd560 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:41.549234 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-0f638479-e54c-4de9-b2cd-929a5a3cf5c7 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:00:41.550431 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-0f638479-e54c-4de9-b2cd-929a5a3cf5c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:00:41.614641 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:41.630771 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 288/564] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:00:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 19:00:41.630771 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f638479-e54c-4de9-b2cd-929a5a3cf5c7 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:41.660049 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af01677f-6fc4-4dff-9fb1-2102d4bbd560 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:41.699103 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc6fffe-ab56-4b4d-a135-3df5dd2d9bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:41.700184 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc6fffe-ab56-4b4d-a135-3df5dd2d9bb6 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:41.700528 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc6fffe-ab56-4b4d-a135-3df5dd2d9bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:41.700809 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc6fffe-ab56-4b4d-a135-3df5dd2d9bb6 None admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:41.701611 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc6fffe-ab56-4b4d-a135-3df5dd2d9bb6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:41.701923 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc6fffe-ab56-4b4d-a135-3df5dd2d9bb6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:41.703428 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc6fffe-ab56-4b4d-a135-3df5dd2d9bb6 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:41.720658 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 277/565] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 2 19:00:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:41.721606 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc6fffe-ab56-4b4d-a135-3df5dd2d9bb6 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:41.734908 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f638479-e54c-4de9-b2cd-929a5a3cf5c7 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:41.753753 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:41.770133 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f4c45d-d4f3-41d0-8f31-693e2a1b43a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:41.770996 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f4c45d-d4f3-41d0-8f31-693e2a1b43a0 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:41.771234 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f4c45d-d4f3-41d0-8f31-693e2a1b43a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:41.771507 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f4c45d-d4f3-41d0-8f31-693e2a1b43a0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:41.779220 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f4c45d-d4f3-41d0-8f31-693e2a1b43a0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:41.780662 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f4c45d-d4f3-41d0-8f31-693e2a1b43a0 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:41.795260 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-58f4c45d-d4f3-41d0-8f31-693e2a1b43a0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 02 19:00:41.795260 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:41.795260 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:41.795260 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:41.795260 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:41.795260 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:41.795260 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:41.795260 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:41.795260 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:41.795260 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:41.795260 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:41.795260 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:41.795260 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 02 19:00:41.795260 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 02 19:00:41.795260 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 02 19:00:41.795260 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 02 19:00:41.796405 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:41.796405 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:41.796405 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 02 19:00:41.796405 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:00:41.796405 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 02 19:00:41.796405 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:00:41.796405 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:41.796405 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:41.796405 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:41.796405 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:41.796405 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:41.796405 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:41.796405 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:41.796405 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:41.796405 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:41.796405 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:41.796405 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:41.796405 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:41.797375 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:41.797375 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:41.797375 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 02 19:00:41.797375 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 02 19:00:41.797375 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 02 19:00:41.797375 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 02 19:00:41.797375 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 02 19:00:41.797375 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 02 19:00:41.797375 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 02 19:00:41.797375 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:00:41.797375 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 289/566] 10.222.0.33 () {56 vars in 1093 bytes} [Thu Jun 2 19:00:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:41.797955 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f4c45d-d4f3-41d0-8f31-693e2a1b43a0 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:41.823954 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc6fffe-ab56-4b4d-a135-3df5dd2d9bb6 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:41.856862 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:41.857618 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ec744b-5797-418c-b8f4-79fdb63c8850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:41.858139 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ec744b-5797-418c-b8f4-79fdb63c8850 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:41.858419 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ec744b-5797-418c-b8f4-79fdb63c8850 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:41.858697 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ec744b-5797-418c-b8f4-79fdb63c8850 None admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:41.859481 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ec744b-5797-418c-b8f4-79fdb63c8850 None admin] RBAC: Adding query filter params (name=service) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:41.859790 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ec744b-5797-418c-b8f4-79fdb63c8850 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:41.861202 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ec744b-5797-418c-b8f4-79fdb63c8850 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:41.869872 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 278/567] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 2 19:00:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:41.870698 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ec744b-5797-418c-b8f4-79fdb63c8850 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:41.906586 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f4c45d-d4f3-41d0-8f31-693e2a1b43a0 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:41.942850 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f9c707-59a8-4132-a23f-d98778c15db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:41.943664 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f9c707-59a8-4132-a23f-d98778c15db7 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:41.943949 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f9c707-59a8-4132-a23f-d98778c15db7 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:41.944226 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f9c707-59a8-4132-a23f-d98778c15db7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:41.954997 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f9c707-59a8-4132-a23f-d98778c15db7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:00:41.962817 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-34f9c707-59a8-4132-a23f-d98778c15db7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 02 19:00:41.962817 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:41.962817 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:41.962817 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:41.962817 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:41.962817 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:41.962817 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:41.962817 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:41.962817 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:41.962817 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:41.962817 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:41.962817 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:41.962817 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:00:41.962817 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:00:41.962817 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:00:41.962817 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:00:41.963718 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:00:41.963718 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:00:41.963718 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:00:41.963718 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:00:41.963718 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:41.963718 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:41.963718 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:41.963718 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:41.963718 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:41.963718 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:41.963718 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:41.963718 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:41.963718 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:41.963718 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:41.963718 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:41.963718 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:41.964534 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:41.964534 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:41.964534 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:41.964534 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:41.964534 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:00:41.964534 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:00:41.964534 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:00:41.964534 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:00:41.964534 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:00:41.964534 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:00:41.964534 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 02 19:00:41.964534 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:00:41.965156 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:41.965230 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 290/568] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 2 19:00:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:41.965230 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9399a2d-8017-482c-b299-8e72ea437aa0 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:41.965516 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9399a2d-8017-482c-b299-8e72ea437aa0 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:41.965793 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9399a2d-8017-482c-b299-8e72ea437aa0 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:41.967080 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 291/569] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:00:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:00:41.967780 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9399a2d-8017-482c-b299-8e72ea437aa0 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:41.979170 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ec744b-5797-418c-b8f4-79fdb63c8850 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:42.014596 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6eea49-9612-427e-8551-699631854ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:42.015211 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6eea49-9612-427e-8551-699631854ad1 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:42.015498 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6eea49-9612-427e-8551-699631854ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:42.015779 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6eea49-9612-427e-8551-699631854ad1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:42.016621 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6eea49-9612-427e-8551-699631854ad1 None admin] RBAC: Adding query filter params (role.id=0e56fba112f94ebba363ffab7f4aa9fe, scope.project.id=8c04020eaebd4647804cf8653f32d9e5, user.id=2a59e1d67ff94186a3ebcfcb49938a87) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:42.016940 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6eea49-9612-427e-8551-699631854ad1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:42.018374 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6eea49-9612-427e-8551-699631854ad1 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:42.024647 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 279/570] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 2 19:00:41 2022] GET /identity/v3/role_assignments?user.id=2a59e1d67ff94186a3ebcfcb49938a87&scope.project.id=8c04020eaebd4647804cf8653f32d9e5&role.id=0e56fba112f94ebba363ffab7f4aa9fe => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:42.025448 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6eea49-9612-427e-8551-699631854ad1 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:42.096686 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9399a2d-8017-482c-b299-8e72ea437aa0 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:42.115960 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:42.139304 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa7dbe7-bc08-4758-a779-a7e49134ae1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:42.140048 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa7dbe7-bc08-4758-a779-a7e49134ae1c None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:42.140333 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa7dbe7-bc08-4758-a779-a7e49134ae1c None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:42.140605 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa7dbe7-bc08-4758-a779-a7e49134ae1c None admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:42.141455 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa7dbe7-bc08-4758-a779-a7e49134ae1c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:42.141765 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa7dbe7-bc08-4758-a779-a7e49134ae1c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:42.143293 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa7dbe7-bc08-4758-a779-a7e49134ae1c None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:42.151857 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6eea49-9612-427e-8551-699631854ad1 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:42.168844 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 292/571] 10.222.0.33 () {56 vars in 1133 bytes} [Thu Jun 2 19:00:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:42.170149 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa7dbe7-bc08-4758-a779-a7e49134ae1c None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:42.192130 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bf72c4-1646-477f-8e56-fc9c8a4638f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:42.192959 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf72c4-1646-477f-8e56-fc9c8a4638f1 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:42.193244 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf72c4-1646-477f-8e56-fc9c8a4638f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:42.193521 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf72c4-1646-477f-8e56-fc9c8a4638f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:42.208075 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bf72c4-1646-477f-8e56-fc9c8a4638f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:42.209629 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bf72c4-1646-477f-8e56-fc9c8a4638f1 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:42.229200 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-d1bf72c4-1646-477f-8e56-fc9c8a4638f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:42.229200 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:42.229200 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:42.229200 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:42.229200 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:42.229200 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:42.229200 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:42.229200 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:42.229200 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:42.229200 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:42.229200 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:42.229200 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:42.229200 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:00:42.229200 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:00:42.229200 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:00:42.230565 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:00:42.230565 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:42.230565 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:42.230565 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:42.230565 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:42.230565 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:42.230565 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:42.230565 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:42.230565 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:42.230565 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:42.230565 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:42.230565 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:42.230565 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:42.230565 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:42.230565 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:42.230565 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:42.230565 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:42.231484 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:00:42.231484 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:00:42.231484 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:42.231484 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:00:42.235240 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 280/572] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:00:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:42.236595 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd8a075-15d3-48b3-bbd8-1fd5b12f3717 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:42.236892 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd8a075-15d3-48b3-bbd8-1fd5b12f3717 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:42.237172 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd8a075-15d3-48b3-bbd8-1fd5b12f3717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:42.255168 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-7bd8a075-15d3-48b3-bbd8-1fd5b12f3717 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:00:42.256322 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-7bd8a075-15d3-48b3-bbd8-1fd5b12f3717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:00:42.259166 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:42.288039 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa7dbe7-bc08-4758-a779-a7e49134ae1c None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:42.325650 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd76112-4bee-412a-8bf8-5021fa7cb529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:42.326327 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd76112-4bee-412a-8bf8-5021fa7cb529 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:42.334615 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd76112-4bee-412a-8bf8-5021fa7cb529 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:42.334811 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 281/573] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:00:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 19:00:42.334925 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd76112-4bee-412a-8bf8-5021fa7cb529 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:42.335003 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd8a075-15d3-48b3-bbd8-1fd5b12f3717 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:42.335075 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd76112-4bee-412a-8bf8-5021fa7cb529 None admin] RBAC: Adding query filter params (name=service) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:42.335075 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd76112-4bee-412a-8bf8-5021fa7cb529 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:42.335075 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd76112-4bee-412a-8bf8-5021fa7cb529 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:42.348594 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 293/574] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 2 19:00:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:42.349828 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd76112-4bee-412a-8bf8-5021fa7cb529 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:42.389320 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:42.442178 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd8a075-15d3-48b3-bbd8-1fd5b12f3717 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:42.457730 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd76112-4bee-412a-8bf8-5021fa7cb529 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:42.479478 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78285233-f7ab-4270-8ed5-0c2230dcd3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:42.480371 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78285233-f7ab-4270-8ed5-0c2230dcd3f9 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:42.480724 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78285233-f7ab-4270-8ed5-0c2230dcd3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:42.481107 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78285233-f7ab-4270-8ed5-0c2230dcd3f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:42.488697 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:42.492946 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78285233-f7ab-4270-8ed5-0c2230dcd3f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:42.494406 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78285233-f7ab-4270-8ed5-0c2230dcd3f9 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:42.496217 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e68352-7447-4091-8427-3421cbb5d32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:42.504252 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e68352-7447-4091-8427-3421cbb5d32a None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:42.504252 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e68352-7447-4091-8427-3421cbb5d32a None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:42.504252 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e68352-7447-4091-8427-3421cbb5d32a None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:42.504252 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e68352-7447-4091-8427-3421cbb5d32a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:42.504252 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e68352-7447-4091-8427-3421cbb5d32a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:42.504252 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e68352-7447-4091-8427-3421cbb5d32a None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:42.519858 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 294/575] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:00:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:42.519858 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e68352-7447-4091-8427-3421cbb5d32a None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:42.520248 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-78285233-f7ab-4270-8ed5-0c2230dcd3f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:42.520248 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:42.520248 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:42.520248 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:42.520248 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:42.520248 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:42.520248 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:42.520248 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:42.520248 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:42.520248 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:42.520248 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:42.520248 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:42.520248 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:00:42.520248 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:00:42.520248 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:00:42.522310 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:00:42.522310 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:42.522310 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:42.522310 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:42.522310 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:42.522310 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:42.522310 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:42.522310 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:42.522310 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:42.522310 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:42.522310 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:42.522310 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:42.522310 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:42.522310 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:42.522310 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:42.522310 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:42.522310 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:42.524750 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:00:42.524750 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:00:42.524750 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:42.524750 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:00:42.524750 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 282/576] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:00:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:42.524750 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78285233-f7ab-4270-8ed5-0c2230dcd3f9 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:42.625096 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:42.628060 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78285233-f7ab-4270-8ed5-0c2230dcd3f9 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:42.633571 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e68352-7447-4091-8427-3421cbb5d32a None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:42.665964 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad1a5c-9571-4807-8b4c-e3147f4984ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:42.666803 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad1a5c-9571-4807-8b4c-e3147f4984ad None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:42.667082 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad1a5c-9571-4807-8b4c-e3147f4984ad None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:42.667386 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad1a5c-9571-4807-8b4c-e3147f4984ad None admin] PATH_INFO: `/v3/roles/service` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:42.668541 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a910d9-20eb-4e87-8564-545df7aee7a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:42.669353 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a910d9-20eb-4e87-8564-545df7aee7a7 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:42.670043 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a910d9-20eb-4e87-8564-545df7aee7a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:42.670495 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a910d9-20eb-4e87-8564-545df7aee7a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:42.678985 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aad1a5c-9571-4807-8b4c-e3147f4984ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:00:42.682171 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a910d9-20eb-4e87-8564-545df7aee7a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:42.683684 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a910d9-20eb-4e87-8564-545df7aee7a7 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:42.687322 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-7aad1a5c-9571-4807-8b4c-e3147f4984ad None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 02 19:00:42.687322 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:42.687322 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:42.687322 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:42.687322 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:42.687322 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:42.687322 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:42.687322 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:42.687322 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:42.687322 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:42.687322 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:42.687322 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:42.687322 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:00:42.687322 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:00:42.687322 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:00:42.687322 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:00:42.688208 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:00:42.688208 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:00:42.688208 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:00:42.688208 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:00:42.688208 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:42.688208 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:42.688208 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:42.688208 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:42.688208 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:42.688208 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:42.688208 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:42.688208 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:42.688208 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:42.688208 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:42.688208 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:42.688208 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:42.689460 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:42.689460 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:42.689460 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:42.689460 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:42.689460 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:00:42.689460 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:00:42.689460 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:00:42.689460 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:00:42.689460 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:00:42.689460 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:00:42.689460 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 02 19:00:42.689460 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:00:42.689460 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 283/577] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 2 19:00:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:42.690327 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad1a5c-9571-4807-8b4c-e3147f4984ad None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:42.707094 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-c0a910d9-20eb-4e87-8564-545df7aee7a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:42.707094 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:42.707094 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:42.707094 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:42.707094 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:42.707094 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:42.707094 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:42.707094 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:42.707094 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:42.707094 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:42.707094 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:42.707094 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:42.707094 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:00:42.707094 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:00:42.707094 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:00:42.708039 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:00:42.708039 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:42.708039 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:42.708039 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:42.708039 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:42.708039 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:42.708039 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:42.708039 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:42.708039 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:42.708039 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:42.708039 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:42.708039 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:42.708039 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:42.708039 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:42.708039 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:42.708039 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:42.708039 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:42.709631 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:00:42.709631 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:00:42.709631 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:42.709631 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:00:42.709631 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 295/578] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:00:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:42.709631 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a910d9-20eb-4e87-8564-545df7aee7a7 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:42.751663 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:42.807723 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad1a5c-9571-4807-8b4c-e3147f4984ad None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:42.814403 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a910d9-20eb-4e87-8564-545df7aee7a7 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:42.853018 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d7350-6361-461a-8bb8-3ccd6af14971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:42.854014 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d7350-6361-461a-8bb8-3ccd6af14971 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:42.854406 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d7350-6361-461a-8bb8-3ccd6af14971 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:42.854691 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d7350-6361-461a-8bb8-3ccd6af14971 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:42.858556 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff005e3-56ad-42e0-9a81-3297d016d627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:42.859480 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff005e3-56ad-42e0-9a81-3297d016d627 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:42.859938 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff005e3-56ad-42e0-9a81-3297d016d627 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:42.860357 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff005e3-56ad-42e0-9a81-3297d016d627 None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:42.861250 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff005e3-56ad-42e0-9a81-3297d016d627 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:42.861701 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff005e3-56ad-42e0-9a81-3297d016d627 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:42.863279 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff005e3-56ad-42e0-9a81-3297d016d627 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:42.863396 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5d7350-6361-461a-8bb8-3ccd6af14971 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:42.864808 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5d7350-6361-461a-8bb8-3ccd6af14971 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:42.881266 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 296/579] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:00:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:42.881266 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff005e3-56ad-42e0-9a81-3297d016d627 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:42.884659 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-3d5d7350-6361-461a-8bb8-3ccd6af14971 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 02 19:00:42.884659 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:42.884659 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:42.884659 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:42.884659 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:42.884659 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:42.884659 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:42.884659 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:42.884659 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:42.884659 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:42.884659 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:42.884659 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:42.884659 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 02 19:00:42.884659 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 02 19:00:42.884659 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 02 19:00:42.884659 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 02 19:00:42.885755 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:42.885755 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:42.885755 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 02 19:00:42.885755 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:00:42.885755 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 02 19:00:42.885755 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:00:42.885755 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:42.885755 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:42.885755 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:42.885755 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:42.885755 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:42.885755 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:42.885755 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:42.885755 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:42.885755 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:42.885755 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:42.885755 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:42.885755 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:42.886824 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:42.886824 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:42.886824 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 02 19:00:42.886824 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 02 19:00:42.886824 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 02 19:00:42.886824 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 02 19:00:42.886824 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 02 19:00:42.886824 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 02 19:00:42.886824 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 02 19:00:42.886824 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:00:42.886824 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 284/580] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 2 19:00:42 2022] GET /identity/v3/users/glance => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:42.888391 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d7350-6361-461a-8bb8-3ccd6af14971 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:42.896459 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:42.991006 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff005e3-56ad-42e0-9a81-3297d016d627 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:43.007215 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d7350-6361-461a-8bb8-3ccd6af14971 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:43.018881 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:43.028340 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e53763-3de2-4ab1-bb8f-f0181bba9b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:43.029015 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e53763-3de2-4ab1-bb8f-f0181bba9b8b None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:43.029300 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e53763-3de2-4ab1-bb8f-f0181bba9b8b None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:43.029578 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e53763-3de2-4ab1-bb8f-f0181bba9b8b None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:43.030326 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e53763-3de2-4ab1-bb8f-f0181bba9b8b None admin] RBAC: Adding query filter params (name=service) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:43.030634 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e53763-3de2-4ab1-bb8f-f0181bba9b8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:43.032143 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e53763-3de2-4ab1-bb8f-f0181bba9b8b None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:43.045497 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 297/581] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 2 19:00:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:43.045497 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e53763-3de2-4ab1-bb8f-f0181bba9b8b None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:43.046530 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea881872-efba-4e4b-8bce-1802629a99f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:43.047232 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea881872-efba-4e4b-8bce-1802629a99f3 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:43.047527 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea881872-efba-4e4b-8bce-1802629a99f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:43.047807 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea881872-efba-4e4b-8bce-1802629a99f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:43.048756 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea881872-efba-4e4b-8bce-1802629a99f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:43.049983 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea881872-efba-4e4b-8bce-1802629a99f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:43.051453 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea881872-efba-4e4b-8bce-1802629a99f3 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:43.071759 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 285/582] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:00:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:43.071759 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea881872-efba-4e4b-8bce-1802629a99f3 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:43.127018 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:43.163188 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e53763-3de2-4ab1-bb8f-f0181bba9b8b None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:43.182713 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea881872-efba-4e4b-8bce-1802629a99f3 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:43.200609 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5089ca91-3f36-42cf-832f-9663e8c7882a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:43.201094 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089ca91-3f36-42cf-832f-9663e8c7882a None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:43.201436 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089ca91-3f36-42cf-832f-9663e8c7882a None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:43.201718 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089ca91-3f36-42cf-832f-9663e8c7882a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:43.209011 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5089ca91-3f36-42cf-832f-9663e8c7882a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:43.210509 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5089ca91-3f36-42cf-832f-9663e8c7882a None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:43.219198 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce473b66-9dce-41fd-86ef-925abad730e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:43.219877 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce473b66-9dce-41fd-86ef-925abad730e8 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:43.220162 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce473b66-9dce-41fd-86ef-925abad730e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:43.220438 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce473b66-9dce-41fd-86ef-925abad730e8 None admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:43.221284 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce473b66-9dce-41fd-86ef-925abad730e8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:43.221594 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce473b66-9dce-41fd-86ef-925abad730e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:43.223084 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce473b66-9dce-41fd-86ef-925abad730e8 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:43.225788 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-5089ca91-3f36-42cf-832f-9663e8c7882a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:00:43.225788 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:43.225788 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:43.225788 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:43.225788 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:43.225788 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:43.225788 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:43.225788 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:43.225788 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:43.225788 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:43.225788 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:43.225788 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:43.225788 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 02 19:00:43.225788 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 02 19:00:43.225788 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 02 19:00:43.227261 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 02 19:00:43.227261 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:43.227261 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:43.227261 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:43.227261 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:43.227261 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:43.227261 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:43.227261 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:43.227261 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:43.227261 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:43.227261 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:43.227261 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:43.227261 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:43.227261 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:43.227261 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:43.227261 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:43.227261 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:43.228127 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 02 19:00:43.228127 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 02 19:00:43.228127 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 02 19:00:43.228127 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 02 19:00:43.228127 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 02 19:00:43.228127 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 02 19:00:43.228127 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:00:43.228127 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:00:43.228127 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 298/583] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 2 19:00:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:43.228600 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5089ca91-3f36-42cf-832f-9663e8c7882a None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:43.245138 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 286/584] 10.222.0.33 () {56 vars in 1137 bytes} [Thu Jun 2 19:00:43 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:43.246623 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce473b66-9dce-41fd-86ef-925abad730e8 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:43.278359 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:43.340417 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5089ca91-3f36-42cf-832f-9663e8c7882a None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:43.353676 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce473b66-9dce-41fd-86ef-925abad730e8 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:43.371526 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:43.378078 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f77c9c-a840-4e6a-9b7d-3e2b160e916b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:43.379416 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f77c9c-a840-4e6a-9b7d-3e2b160e916b None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:43.379957 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f77c9c-a840-4e6a-9b7d-3e2b160e916b None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:43.380533 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f77c9c-a840-4e6a-9b7d-3e2b160e916b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:43.389955 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95408c1f-a5ef-4a81-aa25-18642b3cb652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:43.390798 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95408c1f-a5ef-4a81-aa25-18642b3cb652 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:43.391072 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95408c1f-a5ef-4a81-aa25-18642b3cb652 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:43.391348 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95408c1f-a5ef-4a81-aa25-18642b3cb652 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:43.391464 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f77c9c-a840-4e6a-9b7d-3e2b160e916b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:43.392950 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f77c9c-a840-4e6a-9b7d-3e2b160e916b None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:43.399311 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95408c1f-a5ef-4a81-aa25-18642b3cb652 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:43.400756 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95408c1f-a5ef-4a81-aa25-18642b3cb652 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:43.412466 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-32f77c9c-a840-4e6a-9b7d-3e2b160e916b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:43.412466 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:43.412466 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:43.412466 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:43.412466 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:43.412466 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:43.412466 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:43.412466 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:43.412466 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:43.412466 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:43.412466 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:43.412466 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:43.412466 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:00:43.412466 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:00:43.412466 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:00:43.414144 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:00:43.414144 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:43.414144 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:43.414144 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:43.414144 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:43.414144 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:43.414144 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:43.414144 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:43.414144 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:43.414144 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:43.414144 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:43.414144 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:43.414144 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:43.414144 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:43.414144 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:43.414144 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:43.414144 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:43.415074 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:00:43.415074 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:00:43.415074 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:43.415074 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:00:43.415074 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 299/585] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:00:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:43.415074 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f77c9c-a840-4e6a-9b7d-3e2b160e916b None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:43.419637 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-95408c1f-a5ef-4a81-aa25-18642b3cb652 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 02 19:00:43.419637 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:43.419637 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:43.419637 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:43.419637 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:43.419637 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:43.419637 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:43.419637 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:43.419637 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:43.419637 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:43.419637 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:43.419637 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:43.419637 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 02 19:00:43.419637 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 02 19:00:43.419637 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 02 19:00:43.419637 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 02 19:00:43.420722 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:43.420722 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:43.420722 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 02 19:00:43.420722 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:00:43.420722 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 02 19:00:43.420722 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:00:43.420722 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:43.420722 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:43.420722 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:43.420722 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:43.420722 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:43.420722 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:43.420722 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:43.420722 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:43.420722 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:43.420722 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:43.420722 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:43.420722 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:43.421878 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:43.421878 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:43.421878 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 02 19:00:43.421878 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 02 19:00:43.421878 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 02 19:00:43.421878 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 02 19:00:43.421878 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 02 19:00:43.421878 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 02 19:00:43.421878 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 02 19:00:43.421878 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:00:43.430487 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 287/586] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 2 19:00:43 2022] GET /identity/v3/users/cinder => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:43.430487 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95408c1f-a5ef-4a81-aa25-18642b3cb652 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:43.525933 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:43.539140 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f77c9c-a840-4e6a-9b7d-3e2b160e916b None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:43.540187 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95408c1f-a5ef-4a81-aa25-18642b3cb652 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:43.578490 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1dc733-7b5a-482a-8546-cf9ada723146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:43.578862 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb73fbf-4d97-417b-b9c1-88543a07e7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:43.579199 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1dc733-7b5a-482a-8546-cf9ada723146 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:43.579487 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1dc733-7b5a-482a-8546-cf9ada723146 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:43.579675 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb73fbf-4d97-417b-b9c1-88543a07e7f7 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:43.579770 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1dc733-7b5a-482a-8546-cf9ada723146 None admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:43.579933 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb73fbf-4d97-417b-b9c1-88543a07e7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:43.580210 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb73fbf-4d97-417b-b9c1-88543a07e7f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:43.580554 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1dc733-7b5a-482a-8546-cf9ada723146 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:43.580868 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1dc733-7b5a-482a-8546-cf9ada723146 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:43.582322 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1dc733-7b5a-482a-8546-cf9ada723146 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:43.592587 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb73fbf-4d97-417b-b9c1-88543a07e7f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:43.594283 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb73fbf-4d97-417b-b9c1-88543a07e7f7 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:43.595346 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 300/587] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 2 19:00:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:43.596246 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1dc733-7b5a-482a-8546-cf9ada723146 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:43.619436 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-1fb73fbf-4d97-417b-b9c1-88543a07e7f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:43.619436 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:43.619436 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:43.619436 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:43.619436 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:43.619436 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:43.619436 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:43.619436 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:43.619436 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:43.619436 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:43.619436 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:43.619436 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:43.619436 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:00:43.619436 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:00:43.619436 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:00:43.620557 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:00:43.620557 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:43.620557 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:43.620557 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:43.620557 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:43.620557 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:43.620557 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:43.620557 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:43.620557 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:43.620557 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:43.620557 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:43.620557 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:43.620557 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:43.620557 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:43.620557 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:43.620557 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:43.620557 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:43.621588 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:00:43.621588 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:00:43.621588 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:43.621588 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:00:43.623533 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 288/588] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:00:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:43.624623 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb73fbf-4d97-417b-b9c1-88543a07e7f7 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:43.647602 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:43.705487 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1dc733-7b5a-482a-8546-cf9ada723146 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:43.735071 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb73fbf-4d97-417b-b9c1-88543a07e7f7 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:43.743614 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05028db7-ae4f-4208-a3fa-22844ec7c0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:43.743977 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05028db7-ae4f-4208-a3fa-22844ec7c0f9 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:43.744266 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05028db7-ae4f-4208-a3fa-22844ec7c0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:43.744548 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05028db7-ae4f-4208-a3fa-22844ec7c0f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:43.745290 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05028db7-ae4f-4208-a3fa-22844ec7c0f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:43.745654 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05028db7-ae4f-4208-a3fa-22844ec7c0f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:43.747079 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05028db7-ae4f-4208-a3fa-22844ec7c0f9 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:43.765510 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 301/589] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:00:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:43.765510 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05028db7-ae4f-4208-a3fa-22844ec7c0f9 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:43.783362 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37bd397-0692-4a88-8a39-2b352fa922c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:43.784199 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37bd397-0692-4a88-8a39-2b352fa922c7 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:43.784495 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37bd397-0692-4a88-8a39-2b352fa922c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:43.785078 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37bd397-0692-4a88-8a39-2b352fa922c7 None admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:43.785931 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37bd397-0692-4a88-8a39-2b352fa922c7 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:43.786247 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37bd397-0692-4a88-8a39-2b352fa922c7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:43.788164 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37bd397-0692-4a88-8a39-2b352fa922c7 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:43.799043 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:43.812412 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 289/590] 10.222.0.33 () {56 vars in 1137 bytes} [Thu Jun 2 19:00:43 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:43.812412 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37bd397-0692-4a88-8a39-2b352fa922c7 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:43.875052 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05028db7-ae4f-4208-a3fa-22844ec7c0f9 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:43.877118 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:43.912794 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7f71af-24d9-41eb-a37f-a7470245b497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:43.913722 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7f71af-24d9-41eb-a37f-a7470245b497 None admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:43.914012 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7f71af-24d9-41eb-a37f-a7470245b497 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:43.914290 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7f71af-24d9-41eb-a37f-a7470245b497 None admin] PATH_INFO: `/v3/projects/8c04020eaebd4647804cf8653f32d9e5/users/70664d5cbdee47ee89c120761939cc75/roles/0e56fba112f94ebba363ffab7f4aa9fe` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:43.924102 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37bd397-0692-4a88-8a39-2b352fa922c7 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:43.930043 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7f71af-24d9-41eb-a37f-a7470245b497 None admin] RBAC: Authorizing `identity:create_grant(project_id=8c04020eaebd4647804cf8653f32d9e5, user_id=70664d5cbdee47ee89c120761939cc75, role_id=0e56fba112f94ebba363ffab7f4aa9fe)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:43.931655 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7f71af-24d9-41eb-a37f-a7470245b497 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:43.962020 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aad6771-4988-41b8-bcd0-7a7a31f75d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:43.962628 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad6771-4988-41b8-bcd0-7a7a31f75d7d None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:43.962937 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad6771-4988-41b8-bcd0-7a7a31f75d7d None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:43.963214 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad6771-4988-41b8-bcd0-7a7a31f75d7d None admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:43.963964 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aad6771-4988-41b8-bcd0-7a7a31f75d7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:43.964276 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aad6771-4988-41b8-bcd0-7a7a31f75d7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:43.965686 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aad6771-4988-41b8-bcd0-7a7a31f75d7d None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:43.979765 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 290/591] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:00:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:43.980566 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aad6771-4988-41b8-bcd0-7a7a31f75d7d None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:44.003095 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 302/592] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 2 19:00:43 2022] PUT /identity/v3/projects/8c04020eaebd4647804cf8653f32d9e5/users/70664d5cbdee47ee89c120761939cc75/roles/0e56fba112f94ebba363ffab7f4aa9fe => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:00:44.013911 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7f71af-24d9-41eb-a37f-a7470245b497 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:44.046278 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:44.106734 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aad6771-4988-41b8-bcd0-7a7a31f75d7d None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:44.159499 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4db176-c81d-4f42-b266-bc000ca7ff2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:44.160323 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4db176-c81d-4f42-b266-bc000ca7ff2b None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:44.161006 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4db176-c81d-4f42-b266-bc000ca7ff2b None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:44.161323 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4db176-c81d-4f42-b266-bc000ca7ff2b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:44.173609 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4db176-c81d-4f42-b266-bc000ca7ff2b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:44.175481 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4db176-c81d-4f42-b266-bc000ca7ff2b None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:44.181078 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7f71af-24d9-41eb-a37f-a7470245b497 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:44.196049 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-4b4db176-c81d-4f42-b266-bc000ca7ff2b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 02 19:00:44.196049 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:44.196049 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:44.196049 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:44.196049 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:44.196049 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:44.196049 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:44.196049 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:44.196049 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:44.196049 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:44.196049 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:44.196049 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:44.196049 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 02 19:00:44.196049 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 02 19:00:44.196049 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 02 19:00:44.196049 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 02 19:00:44.197073 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:44.197073 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:44.197073 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 02 19:00:44.197073 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:00:44.197073 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 02 19:00:44.197073 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:00:44.197073 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:44.197073 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:44.197073 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:44.197073 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:44.197073 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:44.197073 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:44.197073 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:44.197073 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:44.197073 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:44.197073 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:44.197073 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:44.197073 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:44.203414 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:44.203414 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:44.203414 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 02 19:00:44.203414 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 02 19:00:44.203414 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 02 19:00:44.203414 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 02 19:00:44.203414 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 02 19:00:44.203414 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 02 19:00:44.203414 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 02 19:00:44.203414 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:00:44.203414 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 291/593] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 2 19:00:43 2022] GET /identity/v3/users/neutron => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:44.203414 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4db176-c81d-4f42-b266-bc000ca7ff2b None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:44.231588 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca1bc40-24c5-4d67-9c3e-f21190a180d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:44.232908 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca1bc40-24c5-4d67-9c3e-f21190a180d7 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:44.233561 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca1bc40-24c5-4d67-9c3e-f21190a180d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:44.234053 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca1bc40-24c5-4d67-9c3e-f21190a180d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:44.248434 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca1bc40-24c5-4d67-9c3e-f21190a180d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:44.250269 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca1bc40-24c5-4d67-9c3e-f21190a180d7 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:44.268594 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-9ca1bc40-24c5-4d67-9c3e-f21190a180d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:44.268594 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:44.268594 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:44.268594 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:44.268594 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:44.268594 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:44.268594 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:44.268594 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:44.268594 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:44.268594 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:44.268594 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:44.268594 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:44.268594 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:00:44.268594 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:00:44.268594 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:00:44.269566 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:00:44.269566 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:44.269566 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:44.269566 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:44.269566 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:44.269566 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:44.269566 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:44.269566 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:44.269566 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:44.269566 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:44.269566 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:44.269566 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:44.269566 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:44.269566 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:44.269566 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:44.269566 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:44.269566 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:44.270658 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:00:44.270658 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:00:44.270658 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:44.270658 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:00:44.271836 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 303/594] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:00:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:44.271836 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca1bc40-24c5-4d67-9c3e-f21190a180d7 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:44.282120 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:44.330291 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4db176-c81d-4f42-b266-bc000ca7ff2b None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:44.347934 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:44.365052 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f465d70-a0dc-4ffd-bd4a-2696364e998d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:44.365857 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f465d70-a0dc-4ffd-bd4a-2696364e998d None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:44.366144 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f465d70-a0dc-4ffd-bd4a-2696364e998d None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:44.366460 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f465d70-a0dc-4ffd-bd4a-2696364e998d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:44.373926 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f465d70-a0dc-4ffd-bd4a-2696364e998d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:44.375438 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f465d70-a0dc-4ffd-bd4a-2696364e998d None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:44.391793 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-2f465d70-a0dc-4ffd-bd4a-2696364e998d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:00:44.391793 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:44.391793 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:44.391793 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:44.391793 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:44.391793 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:44.391793 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:44.391793 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:44.391793 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:44.391793 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:44.391793 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:44.391793 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:44.391793 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 02 19:00:44.391793 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 02 19:00:44.391793 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 02 19:00:44.392898 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 02 19:00:44.392898 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:44.392898 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:44.392898 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:44.392898 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:44.392898 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:44.392898 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:44.392898 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:44.392898 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:44.392898 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:44.392898 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:44.392898 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:44.392898 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:44.392898 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:44.392898 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:44.392898 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:44.392898 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:44.393819 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 02 19:00:44.393819 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 02 19:00:44.393819 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 02 19:00:44.393819 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 02 19:00:44.393819 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 02 19:00:44.393819 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 02 19:00:44.393819 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:00:44.393819 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:00:44.394225 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca1bc40-24c5-4d67-9c3e-f21190a180d7 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:44.394638 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 292/595] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 2 19:00:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:44.394638 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f465d70-a0dc-4ffd-bd4a-2696364e998d None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:44.432909 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd53573-9ad9-41cf-b95a-cb090433a31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:44.433588 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd53573-9ad9-41cf-b95a-cb090433a31c None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:44.433879 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd53573-9ad9-41cf-b95a-cb090433a31c None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:44.434218 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd53573-9ad9-41cf-b95a-cb090433a31c None admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:44.435131 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd53573-9ad9-41cf-b95a-cb090433a31c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:44.435546 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd53573-9ad9-41cf-b95a-cb090433a31c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:44.437002 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd53573-9ad9-41cf-b95a-cb090433a31c None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:44.459317 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 304/596] 10.222.0.33 () {56 vars in 1139 bytes} [Thu Jun 2 19:00:44 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:44.460529 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc8b3ea-e3d2-48fd-88f8-1c140f537770 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:44.460789 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc8b3ea-e3d2-48fd-88f8-1c140f537770 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:44.461070 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc8b3ea-e3d2-48fd-88f8-1c140f537770 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:44.462525 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 305/597] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:00:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:00:44.467754 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc8b3ea-e3d2-48fd-88f8-1c140f537770 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:44.517787 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f465d70-a0dc-4ffd-bd4a-2696364e998d None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:44.536715 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:44.553524 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc34a139-a0cd-4646-84f0-158a4b34ea0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:44.554732 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc34a139-a0cd-4646-84f0-158a4b34ea0d None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:44.555046 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc34a139-a0cd-4646-84f0-158a4b34ea0d None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:44.555322 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc34a139-a0cd-4646-84f0-158a4b34ea0d None admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:44.556056 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc34a139-a0cd-4646-84f0-158a4b34ea0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:44.556360 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc34a139-a0cd-4646-84f0-158a4b34ea0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:44.557752 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc34a139-a0cd-4646-84f0-158a4b34ea0d None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:44.569839 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 293/598] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:00:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:44.571674 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc34a139-a0cd-4646-84f0-158a4b34ea0d None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:44.588737 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc8b3ea-e3d2-48fd-88f8-1c140f537770 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:44.615862 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:44.628214 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d7ccb8-079a-45da-a8ef-de72968f0c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:44.629147 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d7ccb8-079a-45da-a8ef-de72968f0c3b None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:44.629431 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d7ccb8-079a-45da-a8ef-de72968f0c3b None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:44.629784 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d7ccb8-079a-45da-a8ef-de72968f0c3b None admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:44.630603 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d7ccb8-079a-45da-a8ef-de72968f0c3b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:44.630957 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d7ccb8-079a-45da-a8ef-de72968f0c3b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:44.632939 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d7ccb8-079a-45da-a8ef-de72968f0c3b None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:44.651243 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 306/599] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 2 19:00:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:44.651243 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a5f12-a17a-4e0a-8bba-f797375aac37 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:44.651243 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a5f12-a17a-4e0a-8bba-f797375aac37 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:44.651243 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a5f12-a17a-4e0a-8bba-f797375aac37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:44.667145 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-b51a5f12-a17a-4e0a-8bba-f797375aac37 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:00:44.668343 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-b51a5f12-a17a-4e0a-8bba-f797375aac37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:00:44.685016 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc34a139-a0cd-4646-84f0-158a4b34ea0d None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:44.720580 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06154438-b1f0-4baa-b6b3-1300c2c6327d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:44.721385 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06154438-b1f0-4baa-b6b3-1300c2c6327d None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:44.721667 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06154438-b1f0-4baa-b6b3-1300c2c6327d None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:44.721943 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06154438-b1f0-4baa-b6b3-1300c2c6327d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:44.732934 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06154438-b1f0-4baa-b6b3-1300c2c6327d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:44.734740 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06154438-b1f0-4baa-b6b3-1300c2c6327d None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:44.743124 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 307/600] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:00:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 19:00:44.745259 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51a5f12-a17a-4e0a-8bba-f797375aac37 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:44.754151 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-06154438-b1f0-4baa-b6b3-1300c2c6327d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:44.754151 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:44.754151 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:44.754151 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:44.754151 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:44.754151 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:44.754151 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:44.754151 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:44.754151 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:44.754151 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:44.754151 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:44.754151 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:44.754151 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:00:44.754151 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:00:44.754151 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:00:44.762397 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:00:44.762397 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:44.762397 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:44.762397 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:44.762397 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:44.762397 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:44.762397 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:44.762397 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:44.762397 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:44.762397 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:44.762397 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:44.762397 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:44.762397 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:44.762397 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:44.762397 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:44.762397 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:44.762397 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:44.763406 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:00:44.763406 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:00:44.763406 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:44.763406 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:00:44.763406 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 294/601] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:00:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:44.763406 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06154438-b1f0-4baa-b6b3-1300c2c6327d None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:44.811228 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:44.860488 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:44.862810 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51a5f12-a17a-4e0a-8bba-f797375aac37 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:44.870351 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06154438-b1f0-4baa-b6b3-1300c2c6327d None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:44.897532 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1866bc65-2dad-4d6b-81d2-087898e1934b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:44.900235 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1866bc65-2dad-4d6b-81d2-087898e1934b None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:44.900235 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1866bc65-2dad-4d6b-81d2-087898e1934b None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:44.900235 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1866bc65-2dad-4d6b-81d2-087898e1934b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:44.903794 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c5822d-fbdd-4e7a-b759-0fa9e711b7f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:44.904687 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c5822d-fbdd-4e7a-b759-0fa9e711b7f9 None admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:44.904964 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c5822d-fbdd-4e7a-b759-0fa9e711b7f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:44.905235 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c5822d-fbdd-4e7a-b759-0fa9e711b7f9 None admin] PATH_INFO: `/v3/projects/8c04020eaebd4647804cf8653f32d9e5/users/e07892c0aa21449b94a1cc78e3b8a653/roles/0e56fba112f94ebba363ffab7f4aa9fe` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:44.906656 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1866bc65-2dad-4d6b-81d2-087898e1934b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:44.908171 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1866bc65-2dad-4d6b-81d2-087898e1934b None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:44.921162 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c5822d-fbdd-4e7a-b759-0fa9e711b7f9 None admin] RBAC: Authorizing `identity:create_grant(project_id=8c04020eaebd4647804cf8653f32d9e5, user_id=e07892c0aa21449b94a1cc78e3b8a653, role_id=0e56fba112f94ebba363ffab7f4aa9fe)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:44.922613 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c5822d-fbdd-4e7a-b759-0fa9e711b7f9 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:44.924265 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-1866bc65-2dad-4d6b-81d2-087898e1934b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:00:44.924265 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:44.924265 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:44.924265 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:44.924265 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:44.924265 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:44.924265 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:44.924265 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:44.924265 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:44.924265 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:44.924265 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:44.924265 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:44.924265 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 02 19:00:44.924265 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 02 19:00:44.924265 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 02 19:00:44.925140 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 02 19:00:44.925140 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:44.925140 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:44.925140 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:44.925140 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:44.925140 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:44.925140 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:44.925140 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:44.925140 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:44.925140 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:44.925140 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:44.925140 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:44.925140 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:44.925140 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:44.925140 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:44.925140 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:44.925140 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:44.926006 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 02 19:00:44.926006 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 02 19:00:44.926006 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 02 19:00:44.926006 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 02 19:00:44.926006 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 02 19:00:44.926006 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 02 19:00:44.926006 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:00:44.926006 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:00:44.926006 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 308/602] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 2 19:00:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:44.926501 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1866bc65-2dad-4d6b-81d2-087898e1934b None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:44.990829 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 295/603] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 2 19:00:44 2022] PUT /identity/v3/projects/8c04020eaebd4647804cf8653f32d9e5/users/e07892c0aa21449b94a1cc78e3b8a653/roles/0e56fba112f94ebba363ffab7f4aa9fe => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:00:44.992554 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c5822d-fbdd-4e7a-b759-0fa9e711b7f9 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:45.050394 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1866bc65-2dad-4d6b-81d2-087898e1934b None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:45.066042 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:45.103619 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d178c8dd-ef4e-44eb-8b3d-6f61a55de715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:45.104744 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d178c8dd-ef4e-44eb-8b3d-6f61a55de715 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:45.105035 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d178c8dd-ef4e-44eb-8b3d-6f61a55de715 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:45.105317 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d178c8dd-ef4e-44eb-8b3d-6f61a55de715 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:45.128405 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d178c8dd-ef4e-44eb-8b3d-6f61a55de715 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:00:45.133552 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c5822d-fbdd-4e7a-b759-0fa9e711b7f9 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:45.139658 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-d178c8dd-ef4e-44eb-8b3d-6f61a55de715 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 02 19:00:45.139658 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:45.139658 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:45.139658 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:45.139658 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:45.139658 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:45.139658 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:45.139658 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:45.139658 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:45.139658 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:45.139658 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:45.139658 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:45.139658 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:00:45.139658 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:00:45.139658 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:00:45.139658 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:00:45.140681 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:00:45.140681 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:00:45.140681 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:00:45.140681 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:00:45.140681 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:45.140681 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:45.140681 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:45.140681 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:45.140681 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:45.140681 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:45.140681 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:45.140681 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:45.140681 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:45.140681 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:45.140681 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:45.140681 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:45.141496 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:45.141496 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:45.141496 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:45.141496 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:45.141496 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:00:45.141496 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:00:45.141496 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:00:45.141496 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:00:45.141496 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:00:45.141496 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:00:45.141496 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 02 19:00:45.141496 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:00:45.141496 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 309/604] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 2 19:00:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:45.142706 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d178c8dd-ef4e-44eb-8b3d-6f61a55de715 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:45.182517 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395b18fd-9a2c-4882-bcdb-88d2a617141b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:45.183778 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b18fd-9a2c-4882-bcdb-88d2a617141b None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:45.184257 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b18fd-9a2c-4882-bcdb-88d2a617141b None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:45.184736 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b18fd-9a2c-4882-bcdb-88d2a617141b None admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:45.185752 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395b18fd-9a2c-4882-bcdb-88d2a617141b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:45.186372 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395b18fd-9a2c-4882-bcdb-88d2a617141b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:45.188158 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395b18fd-9a2c-4882-bcdb-88d2a617141b None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:45.198876 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:45.210768 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 296/605] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:00:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:45.210768 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395b18fd-9a2c-4882-bcdb-88d2a617141b None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:45.300819 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d178c8dd-ef4e-44eb-8b3d-6f61a55de715 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:45.328158 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395b18fd-9a2c-4882-bcdb-88d2a617141b None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:45.352701 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432a8dad-c4e6-46c6-934a-b59e1c1ad6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:45.354029 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432a8dad-c4e6-46c6-934a-b59e1c1ad6a7 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:45.354997 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432a8dad-c4e6-46c6-934a-b59e1c1ad6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:45.355776 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432a8dad-c4e6-46c6-934a-b59e1c1ad6a7 None admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:45.357050 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432a8dad-c4e6-46c6-934a-b59e1c1ad6a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:45.357992 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432a8dad-c4e6-46c6-934a-b59e1c1ad6a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:45.360316 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432a8dad-c4e6-46c6-934a-b59e1c1ad6a7 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:45.365682 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb11f24-1d68-4be0-9894-bf3683cd5c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:45.366383 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb11f24-1d68-4be0-9894-bf3683cd5c52 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:45.366664 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb11f24-1d68-4be0-9894-bf3683cd5c52 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:45.367696 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb11f24-1d68-4be0-9894-bf3683cd5c52 None admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:45.368453 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb11f24-1d68-4be0-9894-bf3683cd5c52 None admin] RBAC: Adding query filter params (name=service) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:45.368759 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb11f24-1d68-4be0-9894-bf3683cd5c52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:45.370145 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb11f24-1d68-4be0-9894-bf3683cd5c52 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:45.378314 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 310/606] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 2 19:00:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:45.379723 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 297/607] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 2 19:00:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:45.380082 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432a8dad-c4e6-46c6-934a-b59e1c1ad6a7 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:45.384822 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb11f24-1d68-4be0-9894-bf3683cd5c52 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:45.436305 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:45.467152 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:45.501160 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432a8dad-c4e6-46c6-934a-b59e1c1ad6a7 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:45.513891 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb11f24-1d68-4be0-9894-bf3683cd5c52 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:45.537373 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819bfe88-52bc-4712-b9f7-4ca9bcc92ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:45.538236 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819bfe88-52bc-4712-b9f7-4ca9bcc92ac2 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:45.538508 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819bfe88-52bc-4712-b9f7-4ca9bcc92ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:45.539398 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819bfe88-52bc-4712-b9f7-4ca9bcc92ac2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:45.546652 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819bfe88-52bc-4712-b9f7-4ca9bcc92ac2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:45.548662 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819bfe88-52bc-4712-b9f7-4ca9bcc92ac2 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:45.554382 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28ae913-1e13-4ae2-a475-00d9bfdd9bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:45.556241 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ae913-1e13-4ae2-a475-00d9bfdd9bee None admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:45.556732 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ae913-1e13-4ae2-a475-00d9bfdd9bee None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:45.557213 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ae913-1e13-4ae2-a475-00d9bfdd9bee None admin] PATH_INFO: `/v3/projects/8c04020eaebd4647804cf8653f32d9e5/users/091860b2577f49c5af61c75803a90c0d/roles/0e56fba112f94ebba363ffab7f4aa9fe` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:45.564269 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-819bfe88-52bc-4712-b9f7-4ca9bcc92ac2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:00:45.564269 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:45.564269 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:45.564269 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:45.564269 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:45.564269 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:45.564269 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:45.564269 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:45.564269 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:45.564269 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:45.564269 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:45.564269 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:45.564269 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 02 19:00:45.564269 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 02 19:00:45.564269 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 02 19:00:45.565364 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 02 19:00:45.565364 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:45.565364 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:45.565364 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:45.565364 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:45.565364 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:45.565364 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:45.565364 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:45.565364 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:45.565364 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:45.565364 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:45.565364 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:45.565364 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:45.565364 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:45.565364 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:45.565364 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:45.565364 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:45.566336 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 02 19:00:45.566336 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 02 19:00:45.566336 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 02 19:00:45.566336 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 02 19:00:45.566336 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 02 19:00:45.566336 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 02 19:00:45.566336 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:00:45.566336 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:00:45.570633 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 311/608] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 2 19:00:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:45.571766 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819bfe88-52bc-4712-b9f7-4ca9bcc92ac2 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:45.579801 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28ae913-1e13-4ae2-a475-00d9bfdd9bee None admin] RBAC: Authorizing `identity:create_grant(project_id=8c04020eaebd4647804cf8653f32d9e5, user_id=091860b2577f49c5af61c75803a90c0d, role_id=0e56fba112f94ebba363ffab7f4aa9fe)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:45.581665 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28ae913-1e13-4ae2-a475-00d9bfdd9bee None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:45.674817 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:45.690825 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819bfe88-52bc-4712-b9f7-4ca9bcc92ac2 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:45.691316 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 298/609] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 2 19:00:45 2022] PUT /identity/v3/projects/8c04020eaebd4647804cf8653f32d9e5/users/091860b2577f49c5af61c75803a90c0d/roles/0e56fba112f94ebba363ffab7f4aa9fe => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 02 19:00:45.692718 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28ae913-1e13-4ae2-a475-00d9bfdd9bee None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:45.721433 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c979b0-5cb7-4095-bf9a-3d9ced408f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:45.722310 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c979b0-5cb7-4095-bf9a-3d9ced408f4c None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:45.722512 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c979b0-5cb7-4095-bf9a-3d9ced408f4c None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:45.722801 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c979b0-5cb7-4095-bf9a-3d9ced408f4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:45.735409 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c979b0-5cb7-4095-bf9a-3d9ced408f4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:45.736978 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c979b0-5cb7-4095-bf9a-3d9ced408f4c None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:45.758200 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-76c979b0-5cb7-4095-bf9a-3d9ced408f4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:45.758200 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:45.758200 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:45.758200 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:45.758200 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:45.758200 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:45.758200 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:45.758200 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:45.758200 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:45.758200 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:45.758200 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:45.758200 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:45.758200 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:00:45.758200 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:00:45.758200 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:00:45.759238 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:00:45.759238 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:45.759238 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:45.759238 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:45.759238 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:45.759238 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:45.759238 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:45.759238 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:45.759238 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:45.759238 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:45.759238 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:45.759238 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:45.759238 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:45.759238 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:45.759238 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:45.759238 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:45.759238 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:45.760135 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:00:45.760135 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:00:45.760135 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:45.760135 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:00:45.760135 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 312/610] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:00:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:45.768087 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c979b0-5cb7-4095-bf9a-3d9ced408f4c None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:45.882454 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c979b0-5cb7-4095-bf9a-3d9ced408f4c None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:45.921004 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28ae913-1e13-4ae2-a475-00d9bfdd9bee None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:45.923279 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8ff8e6-98a2-406d-a1ab-05397dcdba94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:45.923979 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ff8e6-98a2-406d-a1ab-05397dcdba94 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:45.924330 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ff8e6-98a2-406d-a1ab-05397dcdba94 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:45.924611 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ff8e6-98a2-406d-a1ab-05397dcdba94 None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:45.925448 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8ff8e6-98a2-406d-a1ab-05397dcdba94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:45.925767 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8ff8e6-98a2-406d-a1ab-05397dcdba94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:45.927308 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8ff8e6-98a2-406d-a1ab-05397dcdba94 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:45.947087 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 313/611] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:00:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:45.950703 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8ff8e6-98a2-406d-a1ab-05397dcdba94 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:45.967173 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:45.979180 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:46.016640 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdbde1b-cf58-4c43-b230-cd8b7d0b74a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:46.017452 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdbde1b-cf58-4c43-b230-cd8b7d0b74a5 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:46.017735 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdbde1b-cf58-4c43-b230-cd8b7d0b74a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:46.018011 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdbde1b-cf58-4c43-b230-cd8b7d0b74a5 None admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:46.018850 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdbde1b-cf58-4c43-b230-cd8b7d0b74a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:46.019163 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdbde1b-cf58-4c43-b230-cd8b7d0b74a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:46.020655 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdbde1b-cf58-4c43-b230-cd8b7d0b74a5 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:46.036443 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 299/612] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 2 19:00:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:46.045467 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdbde1b-cf58-4c43-b230-cd8b7d0b74a5 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:46.078289 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8ff8e6-98a2-406d-a1ab-05397dcdba94 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:46.129060 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c35920-79c6-4321-a4ec-30a06e1542a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:46.130285 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c35920-79c6-4321-a4ec-30a06e1542a0 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:46.130649 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c35920-79c6-4321-a4ec-30a06e1542a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:46.131041 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c35920-79c6-4321-a4ec-30a06e1542a0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:46.142493 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c35920-79c6-4321-a4ec-30a06e1542a0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:46.144169 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c35920-79c6-4321-a4ec-30a06e1542a0 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:46.167382 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-65c35920-79c6-4321-a4ec-30a06e1542a0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 02 19:00:46.167382 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:46.167382 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:46.167382 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:46.167382 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:46.167382 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:46.167382 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:46.167382 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:46.167382 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:46.167382 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:46.167382 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:46.167382 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:46.167382 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 02 19:00:46.167382 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 02 19:00:46.167382 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 02 19:00:46.167382 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 02 19:00:46.168649 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:46.168649 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:46.168649 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 02 19:00:46.168649 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:00:46.168649 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 02 19:00:46.168649 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:00:46.168649 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:46.168649 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:46.168649 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:46.168649 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:46.168649 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:46.168649 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:46.168649 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:46.168649 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:46.168649 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:46.168649 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:46.168649 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:46.168649 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:46.169802 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:46.169802 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:46.169802 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 02 19:00:46.169802 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 02 19:00:46.169802 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 02 19:00:46.169802 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 02 19:00:46.169802 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 02 19:00:46.169802 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 02 19:00:46.169802 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 02 19:00:46.169802 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:00:46.170431 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 314/613] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 2 19:00:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:46.176366 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c35920-79c6-4321-a4ec-30a06e1542a0 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:46.191560 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdbde1b-cf58-4c43-b230-cd8b7d0b74a5 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:46.238733 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa3e29e-891a-481d-978b-5586625a9748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:46.239841 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa3e29e-891a-481d-978b-5586625a9748 None admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:46.240131 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa3e29e-891a-481d-978b-5586625a9748 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:46.240410 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa3e29e-891a-481d-978b-5586625a9748 None admin] PATH_INFO: `/v3/projects/8c04020eaebd4647804cf8653f32d9e5/users/7417a54046e34e899d7b03f2f2314deb/roles/0e56fba112f94ebba363ffab7f4aa9fe` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:46.263808 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa3e29e-891a-481d-978b-5586625a9748 None admin] RBAC: Authorizing `identity:create_grant(project_id=8c04020eaebd4647804cf8653f32d9e5, user_id=7417a54046e34e899d7b03f2f2314deb, role_id=0e56fba112f94ebba363ffab7f4aa9fe)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:46.265650 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa3e29e-891a-481d-978b-5586625a9748 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:46.275171 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:46.311001 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c35920-79c6-4321-a4ec-30a06e1542a0 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:46.347490 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca194d5-8905-4598-b04a-1d35ea40acb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:46.348545 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca194d5-8905-4598-b04a-1d35ea40acb3 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:46.349030 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca194d5-8905-4598-b04a-1d35ea40acb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:46.349492 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca194d5-8905-4598-b04a-1d35ea40acb3 None admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:46.350599 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca194d5-8905-4598-b04a-1d35ea40acb3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:46.351672 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca194d5-8905-4598-b04a-1d35ea40acb3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:46.353417 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca194d5-8905-4598-b04a-1d35ea40acb3 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:46.375021 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 315/614] 10.222.0.33 () {56 vars in 1135 bytes} [Thu Jun 2 19:00:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:46.376831 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 300/615] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 2 19:00:46 2022] PUT /identity/v3/projects/8c04020eaebd4647804cf8653f32d9e5/users/7417a54046e34e899d7b03f2f2314deb/roles/0e56fba112f94ebba363ffab7f4aa9fe => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:00:46.383193 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca194d5-8905-4598-b04a-1d35ea40acb3 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:46.495867 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca194d5-8905-4598-b04a-1d35ea40acb3 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:46.522384 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:46.535299 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75d08b2-8842-45f7-8800-fde8f1c9643a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:46.537378 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75d08b2-8842-45f7-8800-fde8f1c9643a None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:46.537378 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75d08b2-8842-45f7-8800-fde8f1c9643a None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:46.537378 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75d08b2-8842-45f7-8800-fde8f1c9643a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:46.547598 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0370d30-f36c-4dbb-a4be-573b1a94b240 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:46.547947 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0370d30-f36c-4dbb-a4be-573b1a94b240 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:46.548171 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0370d30-f36c-4dbb-a4be-573b1a94b240 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:46.548741 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75d08b2-8842-45f7-8800-fde8f1c9643a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:46.550224 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75d08b2-8842-45f7-8800-fde8f1c9643a None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:46.550961 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 301/616] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:00:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:00:46.556101 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a382aa-8191-482f-b585-9746f8f4a0ed None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:46.556417 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a382aa-8191-482f-b585-9746f8f4a0ed None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:46.556674 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a382aa-8191-482f-b585-9746f8f4a0ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:46.573149 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-a75d08b2-8842-45f7-8800-fde8f1c9643a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:46.573149 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:46.573149 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:46.573149 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:46.573149 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:46.573149 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:46.573149 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:46.573149 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:46.573149 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:46.573149 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:46.573149 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:46.573149 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:46.573149 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:00:46.573149 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:00:46.573149 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:00:46.574988 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:00:46.574988 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:46.574988 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:46.574988 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:46.574988 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:46.574988 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:46.574988 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:46.574988 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:46.574988 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:46.574988 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:46.574988 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:46.574988 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:46.574988 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:46.574988 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:46.574988 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:46.574988 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:46.574988 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:46.576083 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:00:46.576083 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:00:46.576083 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:46.576083 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:00:46.576083 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 316/617] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:00:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:46.580066 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75d08b2-8842-45f7-8800-fde8f1c9643a None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:46.585291 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-d8a382aa-8191-482f-b585-9746f8f4a0ed None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:00:46.590997 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-d8a382aa-8191-482f-b585-9746f8f4a0ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:00:46.635188 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:46.727824 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 302/618] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:00:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 02 19:00:46.751936 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75d08b2-8842-45f7-8800-fde8f1c9643a None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:46.752293 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a382aa-8191-482f-b585-9746f8f4a0ed None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:46.798648 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30ad5df-0d37-4771-8b70-95d14dc5efde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:46.799404 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30ad5df-0d37-4771-8b70-95d14dc5efde None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:46.799784 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30ad5df-0d37-4771-8b70-95d14dc5efde None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:46.800060 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30ad5df-0d37-4771-8b70-95d14dc5efde None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:46.800877 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30ad5df-0d37-4771-8b70-95d14dc5efde None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:46.801194 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30ad5df-0d37-4771-8b70-95d14dc5efde None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:46.802576 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30ad5df-0d37-4771-8b70-95d14dc5efde None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:46.817672 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 317/619] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:00:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:46.822795 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30ad5df-0d37-4771-8b70-95d14dc5efde None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:46.886044 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:46.910643 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a382aa-8191-482f-b585-9746f8f4a0ed None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:46.971317 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30ad5df-0d37-4771-8b70-95d14dc5efde None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:47.010480 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ff71a8-e50c-4448-93ee-b964af26ed23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:47.011266 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff71a8-e50c-4448-93ee-b964af26ed23 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:47.011647 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff71a8-e50c-4448-93ee-b964af26ed23 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:47.011935 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff71a8-e50c-4448-93ee-b964af26ed23 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:47.012638 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626e457f-e406-4ede-a353-09e8d4f8c8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:47.013670 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626e457f-e406-4ede-a353-09e8d4f8c8e5 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:47.014108 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626e457f-e406-4ede-a353-09e8d4f8c8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:47.014534 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626e457f-e406-4ede-a353-09e8d4f8c8e5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:47.019907 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ff71a8-e50c-4448-93ee-b964af26ed23 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:47.021544 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ff71a8-e50c-4448-93ee-b964af26ed23 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:47.039320 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-62ff71a8-e50c-4448-93ee-b964af26ed23 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:00:47.039320 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:47.039320 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:47.039320 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:47.039320 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:47.039320 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:47.039320 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:47.039320 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:47.039320 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:47.039320 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:47.039320 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:47.039320 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:47.039320 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 02 19:00:47.039320 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 02 19:00:47.039320 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 02 19:00:47.040463 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 02 19:00:47.040463 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:47.040463 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:47.040463 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:47.040463 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:47.040463 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:47.040463 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:47.040463 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:47.040463 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:47.040463 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:47.040463 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:47.040463 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:47.040463 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:47.040463 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:47.040463 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:47.040463 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:47.040463 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:47.041576 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 02 19:00:47.041576 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 02 19:00:47.041576 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 02 19:00:47.041576 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 02 19:00:47.041576 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 02 19:00:47.041576 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 02 19:00:47.041576 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:00:47.041576 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:00:47.041576 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 318/620] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 2 19:00:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 02 19:00:47.046661 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ff71a8-e50c-4448-93ee-b964af26ed23 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:47.048128 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626e457f-e406-4ede-a353-09e8d4f8c8e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:00:47.056984 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-626e457f-e406-4ede-a353-09e8d4f8c8e5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 02 19:00:47.056984 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:47.056984 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:47.056984 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:47.056984 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:47.056984 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:47.056984 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:47.056984 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:47.056984 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:47.056984 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:47.056984 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:47.056984 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:47.056984 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:00:47.056984 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:00:47.056984 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:00:47.056984 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:00:47.057933 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:00:47.057933 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:00:47.057933 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:00:47.057933 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:00:47.057933 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:47.057933 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:47.057933 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:47.057933 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:47.057933 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:47.057933 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:47.057933 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:47.057933 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:47.057933 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:47.057933 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:47.057933 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:47.057933 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:47.058869 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:47.058869 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:47.058869 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:47.058869 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:47.058869 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:00:47.058869 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:00:47.058869 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:00:47.058869 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:00:47.058869 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:00:47.058869 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:00:47.058869 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 02 19:00:47.058869 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:00:47.058869 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 303/621] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 2 19:00:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 02 19:00:47.065697 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626e457f-e406-4ede-a353-09e8d4f8c8e5 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:47.086481 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:47.183213 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626e457f-e406-4ede-a353-09e8d4f8c8e5 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:47.223760 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c981fc-ccad-44e9-ad56-b8124c91d7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:47.224812 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c981fc-ccad-44e9-ad56-b8124c91d7d9 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:47.225317 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c981fc-ccad-44e9-ad56-b8124c91d7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:47.225816 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c981fc-ccad-44e9-ad56-b8124c91d7d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:47.227403 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c981fc-ccad-44e9-ad56-b8124c91d7d9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:47.227939 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c981fc-ccad-44e9-ad56-b8124c91d7d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:47.229534 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c981fc-ccad-44e9-ad56-b8124c91d7d9 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:47.238928 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 304/622] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 2 19:00:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:47.243122 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:47.245365 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c981fc-ccad-44e9-ad56-b8124c91d7d9 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:47.250883 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ff71a8-e50c-4448-93ee-b964af26ed23 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:47.347217 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:47.349885 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6782329-1a3c-4f3b-b493-c3a5ef129b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:47.350589 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6782329-1a3c-4f3b-b493-c3a5ef129b03 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:47.350907 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6782329-1a3c-4f3b-b493-c3a5ef129b03 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:47.351243 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6782329-1a3c-4f3b-b493-c3a5ef129b03 None admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:47.352567 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6782329-1a3c-4f3b-b493-c3a5ef129b03 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:47.352885 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6782329-1a3c-4f3b-b493-c3a5ef129b03 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:47.354379 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6782329-1a3c-4f3b-b493-c3a5ef129b03 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:47.363318 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c981fc-ccad-44e9-ad56-b8124c91d7d9 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:47.379293 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 319/623] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 2 19:00:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:47.381000 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6782329-1a3c-4f3b-b493-c3a5ef129b03 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:47.430783 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee7569e-b8bd-4b76-b655-d753cc73e30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:47.432383 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7569e-b8bd-4b76-b655-d753cc73e30f None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:47.432899 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7569e-b8bd-4b76-b655-d753cc73e30f None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:47.433389 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7569e-b8bd-4b76-b655-d753cc73e30f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:47.458069 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee7569e-b8bd-4b76-b655-d753cc73e30f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:47.459579 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee7569e-b8bd-4b76-b655-d753cc73e30f None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:47.477925 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-0ee7569e-b8bd-4b76-b655-d753cc73e30f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:47.477925 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:47.477925 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:47.477925 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:47.477925 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:47.477925 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:47.477925 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:47.477925 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:47.477925 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:47.477925 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:47.477925 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:47.477925 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:47.477925 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:00:47.477925 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:00:47.477925 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:00:47.478893 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:00:47.478893 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:47.478893 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:47.478893 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:47.478893 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:47.478893 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:47.478893 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:47.478893 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:47.478893 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:47.478893 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:47.478893 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:47.478893 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:47.478893 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:47.478893 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:47.478893 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:47.478893 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:47.478893 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:47.480188 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:00:47.480188 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:00:47.480188 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:47.480188 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:00:47.480188 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 305/624] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:00:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:47.484932 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee7569e-b8bd-4b76-b655-d753cc73e30f None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:47.523300 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6782329-1a3c-4f3b-b493-c3a5ef129b03 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:47.591875 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee7569e-b8bd-4b76-b655-d753cc73e30f None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:47.595290 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d68ac1-9269-4895-a589-744ca6ff303e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:47.596236 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d68ac1-9269-4895-a589-744ca6ff303e None admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:47.596520 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d68ac1-9269-4895-a589-744ca6ff303e None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:47.596795 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d68ac1-9269-4895-a589-744ca6ff303e None admin] PATH_INFO: `/v3/projects/8c04020eaebd4647804cf8653f32d9e5/users/2a59e1d67ff94186a3ebcfcb49938a87/roles/0e56fba112f94ebba363ffab7f4aa9fe` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:47.619946 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:47.629206 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be69b24f-b986-48d2-bee5-b62dde32108d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:47.629915 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be69b24f-b986-48d2-bee5-b62dde32108d None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:47.633933 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be69b24f-b986-48d2-bee5-b62dde32108d None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:47.633933 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be69b24f-b986-48d2-bee5-b62dde32108d None admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:47.633933 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be69b24f-b986-48d2-bee5-b62dde32108d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:47.633933 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be69b24f-b986-48d2-bee5-b62dde32108d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:47.633933 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be69b24f-b986-48d2-bee5-b62dde32108d None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:47.637874 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d68ac1-9269-4895-a589-744ca6ff303e None admin] RBAC: Authorizing `identity:create_grant(project_id=8c04020eaebd4647804cf8653f32d9e5, user_id=2a59e1d67ff94186a3ebcfcb49938a87, role_id=0e56fba112f94ebba363ffab7f4aa9fe)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:47.639391 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d68ac1-9269-4895-a589-744ca6ff303e None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:47.646258 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 306/625] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:00:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:00:47.652161 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be69b24f-b986-48d2-bee5-b62dde32108d None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:47.716700 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 320/626] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 2 19:00:47 2022] PUT /identity/v3/projects/8c04020eaebd4647804cf8653f32d9e5/users/2a59e1d67ff94186a3ebcfcb49938a87/roles/0e56fba112f94ebba363ffab7f4aa9fe => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:00:47.759987 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be69b24f-b986-48d2-bee5-b62dde32108d None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:47.760358 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77d6c7-8573-4948-8bc6-c16aa561925d None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:47.760905 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77d6c7-8573-4948-8bc6-c16aa561925d None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:47.761435 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77d6c7-8573-4948-8bc6-c16aa561925d None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:47.765047 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 321/627] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:00:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:00:47.770325 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3d1706-5ad4-46cc-b71e-50498e1aeefc None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:47.771032 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3d1706-5ad4-46cc-b71e-50498e1aeefc None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:47.771527 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3d1706-5ad4-46cc-b71e-50498e1aeefc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:47.779067 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:47.803795 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e68aa99-abc6-4a90-86f9-ed279bea9d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:47.805631 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e68aa99-abc6-4a90-86f9-ed279bea9d48 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:47.805745 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-bc3d1706-5ad4-46cc-b71e-50498e1aeefc None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:00:47.806244 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e68aa99-abc6-4a90-86f9-ed279bea9d48 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:47.806739 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e68aa99-abc6-4a90-86f9-ed279bea9d48 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:47.806942 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-bc3d1706-5ad4-46cc-b71e-50498e1aeefc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:00:47.828646 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e68aa99-abc6-4a90-86f9-ed279bea9d48 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:47.830602 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e68aa99-abc6-4a90-86f9-ed279bea9d48 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:47.861305 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-9e68aa99-abc6-4a90-86f9-ed279bea9d48 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 02 19:00:47.861305 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:47.861305 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:47.861305 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:47.861305 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:47.861305 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:47.861305 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:47.861305 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:47.861305 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:47.861305 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:47.861305 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:47.861305 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:47.861305 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 02 19:00:47.861305 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 02 19:00:47.861305 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 02 19:00:47.861305 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 02 19:00:47.862361 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:47.862361 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:47.862361 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 02 19:00:47.862361 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:00:47.862361 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 02 19:00:47.862361 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:00:47.862361 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:47.862361 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:47.862361 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:47.862361 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:47.862361 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:47.862361 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:47.862361 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:47.862361 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:47.862361 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:47.862361 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:47.862361 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:47.862361 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:47.863448 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:47.863448 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:47.863448 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 02 19:00:47.863448 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 02 19:00:47.863448 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 02 19:00:47.863448 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 02 19:00:47.863448 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 02 19:00:47.863448 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 02 19:00:47.863448 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 02 19:00:47.863448 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:00:47.870572 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 307/628] 10.222.0.33 () {56 vars in 1093 bytes} [Thu Jun 2 19:00:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 02 19:00:47.875314 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e68aa99-abc6-4a90-86f9-ed279bea9d48 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:47.974287 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:47.978914 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 322/629] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:00:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 19:00:48.011573 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3d1706-5ad4-46cc-b71e-50498e1aeefc None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:48.036420 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e68aa99-abc6-4a90-86f9-ed279bea9d48 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:48.073585 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f75c477-aca9-4f5b-854d-68805c754dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:48.074272 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f75c477-aca9-4f5b-854d-68805c754dfc None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:48.074557 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f75c477-aca9-4f5b-854d-68805c754dfc None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:48.075303 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f75c477-aca9-4f5b-854d-68805c754dfc None admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:48.075836 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f75c477-aca9-4f5b-854d-68805c754dfc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:48.076149 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f75c477-aca9-4f5b-854d-68805c754dfc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:48.077602 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f75c477-aca9-4f5b-854d-68805c754dfc None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:48.097362 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 308/630] 10.222.0.33 () {56 vars in 1133 bytes} [Thu Jun 2 19:00:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:48.103617 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f75c477-aca9-4f5b-854d-68805c754dfc None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:48.141386 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3d1706-5ad4-46cc-b71e-50498e1aeefc None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:48.209009 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f75c477-aca9-4f5b-854d-68805c754dfc None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:48.212168 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:48.221253 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3681f5ed-0457-432d-90e2-78b1c2827141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:48.222074 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3681f5ed-0457-432d-90e2-78b1c2827141 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:48.222364 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3681f5ed-0457-432d-90e2-78b1c2827141 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:48.222643 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3681f5ed-0457-432d-90e2-78b1c2827141 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:48.228678 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:48.245004 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3681f5ed-0457-432d-90e2-78b1c2827141 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:00:48.246993 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21444a7d-92c0-43f7-9f26-37d4a1ac42d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:48.247710 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21444a7d-92c0-43f7-9f26-37d4a1ac42d9 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:48.248016 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21444a7d-92c0-43f7-9f26-37d4a1ac42d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:48.248291 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21444a7d-92c0-43f7-9f26-37d4a1ac42d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:48.261275 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-3681f5ed-0457-432d-90e2-78b1c2827141 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 02 19:00:48.261275 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:48.261275 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:48.261275 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:48.261275 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:48.261275 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:48.261275 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:48.261275 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:48.261275 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:48.261275 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:48.261275 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:48.261275 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:48.261275 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:00:48.261275 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:00:48.261275 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:00:48.261275 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:00:48.262471 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:00:48.262471 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:00:48.262471 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:00:48.262471 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:00:48.262471 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:48.262471 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:48.262471 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:48.262471 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:48.262471 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:48.262471 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:48.262471 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:48.262471 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:48.262471 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:48.262471 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:48.262471 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:48.262471 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:48.263680 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:48.263680 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:48.263680 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:48.263680 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:48.263680 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:00:48.263680 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:00:48.263680 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:00:48.263680 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:00:48.263680 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:00:48.263680 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:00:48.263680 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 02 19:00:48.263680 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:00:48.263680 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 323/631] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 2 19:00:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:48.271338 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3681f5ed-0457-432d-90e2-78b1c2827141 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:48.322632 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21444a7d-92c0-43f7-9f26-37d4a1ac42d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:48.324164 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21444a7d-92c0-43f7-9f26-37d4a1ac42d9 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:48.487972 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3681f5ed-0457-432d-90e2-78b1c2827141 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:48.493153 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 309/632] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 2 19:00:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 02 19:00:48.498770 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21444a7d-92c0-43f7-9f26-37d4a1ac42d9 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:48.515159 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:48.591530 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e311ca-f1c9-4db5-8262-25dceef46ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:48.592235 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e311ca-f1c9-4db5-8262-25dceef46ac7 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:48.592527 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e311ca-f1c9-4db5-8262-25dceef46ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:48.592812 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e311ca-f1c9-4db5-8262-25dceef46ac7 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:48.593587 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e311ca-f1c9-4db5-8262-25dceef46ac7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:48.593936 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e311ca-f1c9-4db5-8262-25dceef46ac7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:48.595467 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e311ca-f1c9-4db5-8262-25dceef46ac7 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:48.610923 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21444a7d-92c0-43f7-9f26-37d4a1ac42d9 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:48.620141 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 324/633] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 2 19:00:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:48.626114 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e311ca-f1c9-4db5-8262-25dceef46ac7 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:48.653644 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e3ccec-e291-45b0-8808-933f1e5ba753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:48.654640 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e3ccec-e291-45b0-8808-933f1e5ba753 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:48.655059 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e3ccec-e291-45b0-8808-933f1e5ba753 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:48.655428 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e3ccec-e291-45b0-8808-933f1e5ba753 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:48.667901 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e3ccec-e291-45b0-8808-933f1e5ba753 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:48.669729 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e3ccec-e291-45b0-8808-933f1e5ba753 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:48.690206 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-48e3ccec-e291-45b0-8808-933f1e5ba753 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:48.690206 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:48.690206 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:48.690206 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:48.690206 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:48.690206 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:48.690206 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:48.690206 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:48.690206 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:48.690206 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:48.690206 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:48.690206 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:48.690206 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:00:48.690206 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:00:48.690206 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:00:48.691540 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:00:48.691540 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:48.691540 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:48.691540 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:48.691540 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:48.691540 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:48.691540 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:48.691540 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:48.691540 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:48.691540 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:48.691540 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:48.691540 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:48.691540 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:48.691540 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:48.691540 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:48.691540 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:48.691540 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:48.692423 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:00:48.692423 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:00:48.692423 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:48.692423 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:00:48.692654 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 310/634] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:00:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:48.698950 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e3ccec-e291-45b0-8808-933f1e5ba753 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:48.787227 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:48.795119 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:48.879577 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e311ca-f1c9-4db5-8262-25dceef46ac7 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:48.889710 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e3ccec-e291-45b0-8808-933f1e5ba753 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:48.928102 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5911f36-44ec-4679-9471-b6770b40bda7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:48.928453 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeacf72-9149-427d-9fe3-e3d0189f9f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:48.928813 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5911f36-44ec-4679-9471-b6770b40bda7 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:48.929100 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5911f36-44ec-4679-9471-b6770b40bda7 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:48.929202 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeacf72-9149-427d-9fe3-e3d0189f9f8b None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:48.929403 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5911f36-44ec-4679-9471-b6770b40bda7 None admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:48.929559 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeacf72-9149-427d-9fe3-e3d0189f9f8b None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:48.929804 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeacf72-9149-427d-9fe3-e3d0189f9f8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:48.930181 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5911f36-44ec-4679-9471-b6770b40bda7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:48.930492 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5911f36-44ec-4679-9471-b6770b40bda7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:48.932138 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5911f36-44ec-4679-9471-b6770b40bda7 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:48.942560 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeacf72-9149-427d-9fe3-e3d0189f9f8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:48.944123 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeacf72-9149-427d-9fe3-e3d0189f9f8b None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:48.944791 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 311/635] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:00:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:48.950674 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c568cb8-bb82-4923-acce-01f55d398cf8 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:48.950996 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c568cb8-bb82-4923-acce-01f55d398cf8 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:48.951338 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c568cb8-bb82-4923-acce-01f55d398cf8 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:48.953083 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 312/636] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:00:48 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:00:48.953895 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c568cb8-bb82-4923-acce-01f55d398cf8 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:48.964690 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-baeacf72-9149-427d-9fe3-e3d0189f9f8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:48.964690 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:48.964690 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:48.964690 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:48.964690 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:48.964690 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:48.964690 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:48.964690 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:48.964690 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:48.964690 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:48.964690 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:48.964690 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:48.964690 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:00:48.964690 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:00:48.964690 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:00:48.965642 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:00:48.965642 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:48.965642 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:48.965642 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:48.965642 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:48.965642 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:48.965642 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:48.965642 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:48.965642 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:48.965642 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:48.965642 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:48.965642 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:48.965642 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:48.965642 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:48.965642 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:48.965642 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:48.965642 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:48.966517 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:00:48.966517 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:00:48.966517 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:48.966517 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:00:48.966517 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 325/637] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:00:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 02 19:00:48.967850 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac36bf-72a6-49e4-83e8-17132f1dd93f None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:48.968152 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac36bf-72a6-49e4-83e8-17132f1dd93f None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:48.968438 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac36bf-72a6-49e4-83e8-17132f1dd93f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:48.987066 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-7dac36bf-72a6-49e4-83e8-17132f1dd93f None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:00:48.988325 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-7dac36bf-72a6-49e4-83e8-17132f1dd93f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:00:49.064622 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 326/638] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:00:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 19:00:49.065414 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dac36bf-72a6-49e4-83e8-17132f1dd93f None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:49.070984 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c568cb8-bb82-4923-acce-01f55d398cf8 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:49.088891 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:49.108590 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8798711-7e42-40f2-86bb-23dc157ac068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:49.109400 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8798711-7e42-40f2-86bb-23dc157ac068 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:49.109948 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:49.110154 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8798711-7e42-40f2-86bb-23dc157ac068 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:49.110462 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8798711-7e42-40f2-86bb-23dc157ac068 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:49.122648 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8798711-7e42-40f2-86bb-23dc157ac068 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:49.123988 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8798711-7e42-40f2-86bb-23dc157ac068 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:49.142526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-d8798711-7e42-40f2-86bb-23dc157ac068 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:00:49.142526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:49.142526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:49.142526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:49.142526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:49.142526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:49.142526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:49.142526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:49.142526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:49.142526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:49.142526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:49.142526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:49.142526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 02 19:00:49.142526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 02 19:00:49.142526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 02 19:00:49.150043 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 02 19:00:49.150043 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:49.150043 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:49.150043 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:49.150043 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:49.150043 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:49.150043 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:49.150043 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:49.150043 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:49.150043 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:49.150043 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:49.150043 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:49.150043 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:49.150043 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:49.150043 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:49.150043 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:49.150043 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:49.150936 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 02 19:00:49.150936 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 02 19:00:49.150936 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 02 19:00:49.150936 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 02 19:00:49.150936 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 02 19:00:49.150936 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 02 19:00:49.150936 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:00:49.150936 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:00:49.150936 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 313/639] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 2 19:00:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:49.150936 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a959644-dee9-48e7-96ff-f419ba313b6b None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:49.150936 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a959644-dee9-48e7-96ff-f419ba313b6b None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:49.151528 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a959644-dee9-48e7-96ff-f419ba313b6b None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:49.151528 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 314/640] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:00:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:00:49.151528 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a959644-dee9-48e7-96ff-f419ba313b6b None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:49.184147 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dac36bf-72a6-49e4-83e8-17132f1dd93f None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:49.225401 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c151b8-b9d2-407d-b324-5c2051ef5446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:49.226075 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c151b8-b9d2-407d-b324-5c2051ef5446 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:49.227008 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c151b8-b9d2-407d-b324-5c2051ef5446 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:49.227288 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c151b8-b9d2-407d-b324-5c2051ef5446 None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:49.228024 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c151b8-b9d2-407d-b324-5c2051ef5446 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:49.228333 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c151b8-b9d2-407d-b324-5c2051ef5446 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:49.229712 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c151b8-b9d2-407d-b324-5c2051ef5446 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:49.245153 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 327/641] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:00:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:49.245153 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c151b8-b9d2-407d-b324-5c2051ef5446 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:49.275286 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a959644-dee9-48e7-96ff-f419ba313b6b None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:49.314057 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3c2f47-a8b8-4c04-ab0e-7bf39fe1e015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:49.314940 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3c2f47-a8b8-4c04-ab0e-7bf39fe1e015 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:49.315163 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3c2f47-a8b8-4c04-ab0e-7bf39fe1e015 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:49.315540 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3c2f47-a8b8-4c04-ab0e-7bf39fe1e015 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:49.326520 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3c2f47-a8b8-4c04-ab0e-7bf39fe1e015 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:00:49.334420 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-cd3c2f47-a8b8-4c04-ab0e-7bf39fe1e015 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 02 19:00:49.334420 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:49.334420 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:49.334420 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:49.334420 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:49.334420 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:49.334420 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:49.334420 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:49.334420 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:49.334420 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:49.334420 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:49.334420 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:49.334420 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:00:49.334420 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:00:49.334420 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:00:49.334420 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:00:49.335715 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:00:49.335715 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:00:49.335715 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:00:49.335715 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:00:49.335715 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:49.335715 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:49.335715 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:49.335715 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:49.335715 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:49.335715 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:49.335715 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:49.335715 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:49.335715 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:49.335715 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:49.335715 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:49.335715 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:49.337185 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:49.337185 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:49.337185 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:49.337185 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:49.337185 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:00:49.337185 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:00:49.337185 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:00:49.337185 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:00:49.337185 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:00:49.337185 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:00:49.337185 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 02 19:00:49.337185 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:00:49.337185 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 315/642] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 2 19:00:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:49.342365 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76da7b73-6320-4c7c-a63c-f654bf9bd93a None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:49.342657 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76da7b73-6320-4c7c-a63c-f654bf9bd93a None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:49.342958 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76da7b73-6320-4c7c-a63c-f654bf9bd93a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:49.362319 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-76da7b73-6320-4c7c-a63c-f654bf9bd93a None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:00:49.363553 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-76da7b73-6320-4c7c-a63c-f654bf9bd93a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:00:49.364954 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:49.369704 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c151b8-b9d2-407d-b324-5c2051ef5446 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:49.396032 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:49.415647 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c0952d-9df7-43ce-a782-533090830787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:49.416420 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c0952d-9df7-43ce-a782-533090830787 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:49.416718 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c0952d-9df7-43ce-a782-533090830787 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:49.416996 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c0952d-9df7-43ce-a782-533090830787 None admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:49.417791 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c0952d-9df7-43ce-a782-533090830787 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:49.418113 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c0952d-9df7-43ce-a782-533090830787 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:49.421664 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c0952d-9df7-43ce-a782-533090830787 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:49.439425 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 328/643] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 2 19:00:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:49.439425 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c0952d-9df7-43ce-a782-533090830787 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:49.441797 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 316/644] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:00:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 19:00:49.442742 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76da7b73-6320-4c7c-a63c-f654bf9bd93a None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:49.560958 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76da7b73-6320-4c7c-a63c-f654bf9bd93a None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:49.562130 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c0952d-9df7-43ce-a782-533090830787 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:49.596923 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ae281f-bbf3-4359-a286-245cb99fc678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:49.597223 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae796afa-ea14-49e5-9888-013cf82d4e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:49.597674 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae281f-bbf3-4359-a286-245cb99fc678 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:49.597839 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae281f-bbf3-4359-a286-245cb99fc678 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:49.598048 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae796afa-ea14-49e5-9888-013cf82d4e8a None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:49.598158 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae281f-bbf3-4359-a286-245cb99fc678 None admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:49.598334 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae796afa-ea14-49e5-9888-013cf82d4e8a None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:49.598607 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae796afa-ea14-49e5-9888-013cf82d4e8a None admin] PATH_INFO: `/v3/users/glance` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:49.598898 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ae281f-bbf3-4359-a286-245cb99fc678 None admin] RBAC: Adding query filter params (name=service) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:49.599212 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ae281f-bbf3-4359-a286-245cb99fc678 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:49.600621 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ae281f-bbf3-4359-a286-245cb99fc678 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:49.606141 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae796afa-ea14-49e5-9888-013cf82d4e8a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:49.607607 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae796afa-ea14-49e5-9888-013cf82d4e8a None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:49.613089 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 317/645] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 2 19:00:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:49.613089 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ae281f-bbf3-4359-a286-245cb99fc678 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:49.628965 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-ae796afa-ea14-49e5-9888-013cf82d4e8a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 02 19:00:49.628965 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:49.628965 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:49.628965 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:49.628965 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:49.628965 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:49.628965 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:49.628965 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:49.628965 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:49.628965 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:49.628965 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:49.628965 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:49.628965 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 02 19:00:49.628965 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 02 19:00:49.628965 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 02 19:00:49.628965 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 02 19:00:49.630003 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:49.630003 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:49.630003 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 02 19:00:49.630003 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:00:49.630003 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 02 19:00:49.630003 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:00:49.630003 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:49.630003 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:49.630003 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:49.630003 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:49.630003 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:49.630003 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:49.630003 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:49.630003 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:49.630003 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:49.630003 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:49.630003 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:49.630003 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:49.631622 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:49.631622 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:49.631622 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 02 19:00:49.631622 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 02 19:00:49.631622 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 02 19:00:49.631622 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 02 19:00:49.631622 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 02 19:00:49.631622 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 02 19:00:49.631622 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 02 19:00:49.631622 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:00:49.631622 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 329/646] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 2 19:00:49 2022] GET /identity/v3/users/glance => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:49.631622 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae796afa-ea14-49e5-9888-013cf82d4e8a None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:49.646642 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:49.652182 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:49.720360 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ae281f-bbf3-4359-a286-245cb99fc678 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:49.739178 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae796afa-ea14-49e5-9888-013cf82d4e8a None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:49.761100 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8376c13-9fca-47a0-b7cb-69142b557e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:49.761765 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8376c13-9fca-47a0-b7cb-69142b557e86 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:49.762047 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8376c13-9fca-47a0-b7cb-69142b557e86 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:49.762322 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8376c13-9fca-47a0-b7cb-69142b557e86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:49.763215 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8376c13-9fca-47a0-b7cb-69142b557e86 None admin] RBAC: Adding query filter params (role.id=0e56fba112f94ebba363ffab7f4aa9fe, scope.project.id=8c04020eaebd4647804cf8653f32d9e5, user.id=70664d5cbdee47ee89c120761939cc75) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:49.763528 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8376c13-9fca-47a0-b7cb-69142b557e86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:49.764956 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8376c13-9fca-47a0-b7cb-69142b557e86 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:49.771898 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 318/647] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 2 19:00:49 2022] GET /identity/v3/role_assignments?user.id=70664d5cbdee47ee89c120761939cc75&scope.project.id=8c04020eaebd4647804cf8653f32d9e5&role.id=0e56fba112f94ebba363ffab7f4aa9fe => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:49.778465 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7df511f-5305-4d61-a159-83247af2190a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:49.779323 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df511f-5305-4d61-a159-83247af2190a None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:49.779608 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df511f-5305-4d61-a159-83247af2190a None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:49.779883 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df511f-5305-4d61-a159-83247af2190a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:49.784308 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8376c13-9fca-47a0-b7cb-69142b557e86 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:49.793415 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7df511f-5305-4d61-a159-83247af2190a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:00:49.801593 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-c7df511f-5305-4d61-a159-83247af2190a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 02 19:00:49.801593 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:49.801593 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:49.801593 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:49.801593 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:49.801593 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:49.801593 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:49.801593 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:49.801593 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:49.801593 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:49.801593 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:49.801593 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:49.801593 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:00:49.801593 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:00:49.801593 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:00:49.801593 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:00:49.802594 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:00:49.802594 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:00:49.802594 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:00:49.802594 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:00:49.802594 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:49.802594 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:49.802594 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:49.802594 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:49.802594 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:49.802594 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:49.802594 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:49.802594 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:49.802594 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:49.802594 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:49.802594 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:49.802594 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:49.803467 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:49.803467 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:49.803467 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:49.803467 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:49.803467 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:00:49.803467 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:00:49.803467 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:00:49.803467 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:00:49.803467 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:00:49.803467 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:00:49.803467 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 02 19:00:49.803467 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:00:49.807189 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 330/648] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 2 19:00:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:49.808203 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7df511f-5305-4d61-a159-83247af2190a None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:49.916660 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:49.923407 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:49.934558 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8376c13-9fca-47a0-b7cb-69142b557e86 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:49.948573 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7df511f-5305-4d61-a159-83247af2190a None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:49.982779 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c838fec-4979-46d6-991a-639eb19c86d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:49.983398 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c838fec-4979-46d6-991a-639eb19c86d8 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:49.983781 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c838fec-4979-46d6-991a-639eb19c86d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:49.984061 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c838fec-4979-46d6-991a-639eb19c86d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:49.996740 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c838fec-4979-46d6-991a-639eb19c86d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:49.997373 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d995794b-6893-4d30-af55-d03576b0310d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:49.998205 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c838fec-4979-46d6-991a-639eb19c86d8 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:49.998399 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d995794b-6893-4d30-af55-d03576b0310d None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:49.998854 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d995794b-6893-4d30-af55-d03576b0310d None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:49.999289 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d995794b-6893-4d30-af55-d03576b0310d None admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:50.000412 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d995794b-6893-4d30-af55-d03576b0310d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:50.000993 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d995794b-6893-4d30-af55-d03576b0310d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:50.002608 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d995794b-6893-4d30-af55-d03576b0310d None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:50.018275 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-3c838fec-4979-46d6-991a-639eb19c86d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:50.018275 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:50.018275 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:50.018275 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:50.018275 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:50.018275 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:50.018275 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:50.018275 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:50.018275 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:50.018275 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:50.018275 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:50.018275 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:50.018275 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:00:50.018275 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:00:50.018275 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:00:50.019257 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:00:50.019257 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:50.019257 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:50.019257 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:50.019257 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:50.019257 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:50.019257 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:50.019257 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:50.019257 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:50.019257 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:50.019257 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:50.019257 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:50.019257 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:50.019257 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:50.019257 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:50.019257 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:50.019257 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:50.020138 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:00:50.020138 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:00:50.020138 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:50.020138 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:00:50.020138 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 319/649] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:00:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:50.020776 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c838fec-4979-46d6-991a-639eb19c86d8 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:50.026652 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 331/650] 10.222.0.33 () {56 vars in 1137 bytes} [Thu Jun 2 19:00:49 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:50.032621 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d995794b-6893-4d30-af55-d03576b0310d None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:50.161190 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c838fec-4979-46d6-991a-639eb19c86d8 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:50.162683 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d995794b-6893-4d30-af55-d03576b0310d None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:50.205296 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64c5797-0ae4-47e4-98b7-0eaa39e390f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:50.205496 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884264c6-625b-41b3-98ed-4aec34ed24ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:50.206086 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884264c6-625b-41b3-98ed-4aec34ed24ba None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:50.206373 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884264c6-625b-41b3-98ed-4aec34ed24ba None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:50.206623 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64c5797-0ae4-47e4-98b7-0eaa39e390f9 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:50.206732 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884264c6-625b-41b3-98ed-4aec34ed24ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:50.207151 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64c5797-0ae4-47e4-98b7-0eaa39e390f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:50.207658 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64c5797-0ae4-47e4-98b7-0eaa39e390f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:50.208560 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64c5797-0ae4-47e4-98b7-0eaa39e390f9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:50.209007 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64c5797-0ae4-47e4-98b7-0eaa39e390f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:50.211451 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64c5797-0ae4-47e4-98b7-0eaa39e390f9 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:50.211878 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:50.224271 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 320/651] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 2 19:00:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:50.228814 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64c5797-0ae4-47e4-98b7-0eaa39e390f9 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:50.238357 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:50.292980 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884264c6-625b-41b3-98ed-4aec34ed24ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:50.295523 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884264c6-625b-41b3-98ed-4aec34ed24ba None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:50.355200 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64c5797-0ae4-47e4-98b7-0eaa39e390f9 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:50.391302 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e632d601-41fa-4a92-8f1e-b3f3fe7ad767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:50.391968 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e632d601-41fa-4a92-8f1e-b3f3fe7ad767 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:50.392251 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e632d601-41fa-4a92-8f1e-b3f3fe7ad767 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:50.392526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e632d601-41fa-4a92-8f1e-b3f3fe7ad767 None admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:50.393257 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e632d601-41fa-4a92-8f1e-b3f3fe7ad767 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:50.393563 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e632d601-41fa-4a92-8f1e-b3f3fe7ad767 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:50.394988 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e632d601-41fa-4a92-8f1e-b3f3fe7ad767 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:50.411814 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 321/652] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:00:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:50.411814 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e632d601-41fa-4a92-8f1e-b3f3fe7ad767 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:50.428754 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 332/653] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 2 19:00:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 02 19:00:50.430423 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884264c6-625b-41b3-98ed-4aec34ed24ba None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:50.468074 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:50.483478 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:50.523978 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e632d601-41fa-4a92-8f1e-b3f3fe7ad767 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:50.536024 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884264c6-625b-41b3-98ed-4aec34ed24ba None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:50.559322 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7896cb8-a80c-401a-a28f-c43bd2ed9e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:50.560125 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7896cb8-a80c-401a-a28f-c43bd2ed9e5e None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:50.560528 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7896cb8-a80c-401a-a28f-c43bd2ed9e5e None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:50.560710 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7896cb8-a80c-401a-a28f-c43bd2ed9e5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:50.571993 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7896cb8-a80c-401a-a28f-c43bd2ed9e5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:50.573379 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e472d82-bb16-433c-82a2-bdde510d9e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:50.573526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7896cb8-a80c-401a-a28f-c43bd2ed9e5e None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:50.574177 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e472d82-bb16-433c-82a2-bdde510d9e93 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:50.574458 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e472d82-bb16-433c-82a2-bdde510d9e93 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:50.574731 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e472d82-bb16-433c-82a2-bdde510d9e93 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:50.581976 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e472d82-bb16-433c-82a2-bdde510d9e93 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:50.583454 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e472d82-bb16-433c-82a2-bdde510d9e93 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:50.590784 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-a7896cb8-a80c-401a-a28f-c43bd2ed9e5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:50.590784 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:50.590784 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:50.590784 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:50.590784 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:50.590784 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:50.590784 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:50.590784 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:50.590784 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:50.590784 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:50.590784 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:50.590784 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:50.590784 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:00:50.590784 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:00:50.590784 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:00:50.591644 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:00:50.591644 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:50.591644 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:50.591644 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:50.591644 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:50.591644 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:50.591644 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:50.591644 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:50.591644 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:50.591644 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:50.591644 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:50.591644 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:50.591644 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:50.591644 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:50.591644 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:50.591644 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:50.591644 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:50.592618 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:00:50.592618 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:00:50.592618 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:50.592618 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:00:50.595169 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 322/654] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:00:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:50.596288 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7896cb8-a80c-401a-a28f-c43bd2ed9e5e None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:50.598175 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-8e472d82-bb16-433c-82a2-bdde510d9e93 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 02 19:00:50.598175 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:50.598175 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:50.598175 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:50.598175 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:50.598175 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:50.598175 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:50.598175 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:50.598175 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:50.598175 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:50.598175 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:50.598175 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:50.598175 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 02 19:00:50.598175 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 02 19:00:50.598175 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 02 19:00:50.598175 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 02 19:00:50.599626 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:50.599626 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:50.599626 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 02 19:00:50.599626 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:00:50.599626 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 02 19:00:50.599626 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:00:50.599626 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:50.599626 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:50.599626 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:50.599626 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:50.599626 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:50.599626 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:50.599626 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:50.599626 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:50.599626 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:50.599626 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:50.599626 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:50.599626 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:50.601045 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:50.601045 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:50.601045 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 02 19:00:50.601045 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 02 19:00:50.601045 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 02 19:00:50.601045 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 02 19:00:50.601045 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 02 19:00:50.601045 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 02 19:00:50.601045 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 02 19:00:50.601045 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:00:50.601045 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 333/655] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 2 19:00:50 2022] GET /identity/v3/users/cinder => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:50.601045 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36fd3de-29b8-4039-8adb-75c9ba0a2c08 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:50.602258 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36fd3de-29b8-4039-8adb-75c9ba0a2c08 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:50.602258 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36fd3de-29b8-4039-8adb-75c9ba0a2c08 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:50.607898 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 334/656] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:00:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:00:50.607898 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36fd3de-29b8-4039-8adb-75c9ba0a2c08 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:50.715421 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:50.718697 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7896cb8-a80c-401a-a28f-c43bd2ed9e5e None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:50.723027 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36fd3de-29b8-4039-8adb-75c9ba0a2c08 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:50.737741 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:50.757378 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55f3e8f-5364-4681-981f-4c52f72c7c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:50.758942 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55f3e8f-5364-4681-981f-4c52f72c7c95 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:50.760843 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55f3e8f-5364-4681-981f-4c52f72c7c95 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:50.760843 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55f3e8f-5364-4681-981f-4c52f72c7c95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:50.763281 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2572d747-32cb-4528-a8ff-624eadf6dea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:50.763987 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2572d747-32cb-4528-a8ff-624eadf6dea8 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:50.764273 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2572d747-32cb-4528-a8ff-624eadf6dea8 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:50.764552 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2572d747-32cb-4528-a8ff-624eadf6dea8 None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:50.765289 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2572d747-32cb-4528-a8ff-624eadf6dea8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:50.765602 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2572d747-32cb-4528-a8ff-624eadf6dea8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:50.767375 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2572d747-32cb-4528-a8ff-624eadf6dea8 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:50.770947 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55f3e8f-5364-4681-981f-4c52f72c7c95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:50.772547 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55f3e8f-5364-4681-981f-4c52f72c7c95 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:50.779927 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 335/657] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:00:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:50.780790 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2572d747-32cb-4528-a8ff-624eadf6dea8 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:50.795529 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-b55f3e8f-5364-4681-981f-4c52f72c7c95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:50.795529 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:50.795529 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:50.795529 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:50.795529 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:50.795529 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:50.795529 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:50.795529 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:50.795529 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:50.795529 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:50.795529 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:50.795529 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:50.795529 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:00:50.795529 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:00:50.795529 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:00:50.796456 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:00:50.796456 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:50.796456 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:50.796456 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:50.796456 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:50.796456 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:50.796456 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:50.796456 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:50.796456 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:50.796456 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:50.796456 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:50.796456 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:50.796456 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:50.796456 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:50.796456 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:50.796456 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:50.796456 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:50.797316 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:00:50.797316 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:00:50.797316 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:50.797316 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:00:50.797316 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 323/658] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:00:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:50.798076 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8278d8e-2b8f-47a5-8ba0-a2682b749af8 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:50.798238 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8278d8e-2b8f-47a5-8ba0-a2682b749af8 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:50.798502 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8278d8e-2b8f-47a5-8ba0-a2682b749af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:50.815487 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-a8278d8e-2b8f-47a5-8ba0-a2682b749af8 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:00:50.816665 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-a8278d8e-2b8f-47a5-8ba0-a2682b749af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:00:50.889345 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 324/659] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:00:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 19:00:50.890221 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8278d8e-2b8f-47a5-8ba0-a2682b749af8 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:50.902337 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2572d747-32cb-4528-a8ff-624eadf6dea8 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:50.952295 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af1905a-e9cc-4321-b61f-5e88fdf130d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:50.953600 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af1905a-e9cc-4321-b61f-5e88fdf130d9 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:50.953986 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af1905a-e9cc-4321-b61f-5e88fdf130d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:50.954438 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af1905a-e9cc-4321-b61f-5e88fdf130d9 None admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:50.956086 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af1905a-e9cc-4321-b61f-5e88fdf130d9 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:50.956575 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af1905a-e9cc-4321-b61f-5e88fdf130d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:50.959169 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af1905a-e9cc-4321-b61f-5e88fdf130d9 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:50.991008 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 336/660] 10.222.0.33 () {56 vars in 1137 bytes} [Thu Jun 2 19:00:50 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:50.992885 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af1905a-e9cc-4321-b61f-5e88fdf130d9 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:51.012696 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:51.042385 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:51.045410 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8278d8e-2b8f-47a5-8ba0-a2682b749af8 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:51.091372 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac85938-c390-427d-8b46-559d38bf2b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:51.092725 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac85938-c390-427d-8b46-559d38bf2b5d None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:51.093135 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac85938-c390-427d-8b46-559d38bf2b5d None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:51.093498 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac85938-c390-427d-8b46-559d38bf2b5d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:51.104679 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac85938-c390-427d-8b46-559d38bf2b5d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:51.106917 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac85938-c390-427d-8b46-559d38bf2b5d None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:51.114060 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af1905a-e9cc-4321-b61f-5e88fdf130d9 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:51.126150 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-dac85938-c390-427d-8b46-559d38bf2b5d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 02 19:00:51.126150 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:51.126150 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:51.126150 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:51.126150 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:51.126150 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:51.126150 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:51.126150 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:51.126150 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:51.126150 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:51.126150 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:51.126150 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:51.126150 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 02 19:00:51.126150 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 02 19:00:51.126150 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 02 19:00:51.126150 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 02 19:00:51.127301 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:51.127301 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:51.127301 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 02 19:00:51.127301 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:00:51.127301 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 02 19:00:51.127301 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:00:51.127301 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:51.127301 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:51.127301 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:51.127301 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:51.127301 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:51.127301 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:51.127301 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:51.127301 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:51.127301 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:51.127301 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:51.127301 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:51.127301 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:51.128236 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:51.128236 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:51.128236 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 02 19:00:51.128236 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 02 19:00:51.128236 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 02 19:00:51.128236 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 02 19:00:51.128236 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 02 19:00:51.128236 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 02 19:00:51.128236 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 02 19:00:51.128236 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:00:51.128236 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 325/661] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 2 19:00:50 2022] GET /identity/v3/users/neutron => generated 85 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:51.129100 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac85938-c390-427d-8b46-559d38bf2b5d None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:51.160189 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a249a315-1427-4b4a-82fe-a0f9b636ecc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:51.160888 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a249a315-1427-4b4a-82fe-a0f9b636ecc6 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:51.161210 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a249a315-1427-4b4a-82fe-a0f9b636ecc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:51.161473 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a249a315-1427-4b4a-82fe-a0f9b636ecc6 None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:51.162234 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a249a315-1427-4b4a-82fe-a0f9b636ecc6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:51.162562 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a249a315-1427-4b4a-82fe-a0f9b636ecc6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:51.164015 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a249a315-1427-4b4a-82fe-a0f9b636ecc6 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:51.176868 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 337/662] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:00:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:51.177466 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a249a315-1427-4b4a-82fe-a0f9b636ecc6 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:51.255906 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac85938-c390-427d-8b46-559d38bf2b5d None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:51.265427 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:51.292453 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a249a315-1427-4b4a-82fe-a0f9b636ecc6 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:51.296648 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5126d3-470b-4ca5-b1a2-ea6b12eb369e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:51.297686 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5126d3-470b-4ca5-b1a2-ea6b12eb369e None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:51.297816 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5126d3-470b-4ca5-b1a2-ea6b12eb369e None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:51.298014 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5126d3-470b-4ca5-b1a2-ea6b12eb369e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:51.313270 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5126d3-470b-4ca5-b1a2-ea6b12eb369e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:00:51.319856 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-1f5126d3-470b-4ca5-b1a2-ea6b12eb369e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 02 19:00:51.319856 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:51.319856 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:51.319856 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:51.319856 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:51.319856 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:51.319856 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:51.319856 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:51.319856 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:51.319856 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:51.319856 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:51.319856 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:51.319856 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:00:51.319856 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:00:51.319856 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:00:51.319856 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:00:51.323778 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:00:51.323778 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:00:51.323778 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:00:51.323778 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:00:51.323778 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:51.323778 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:51.323778 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:51.323778 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:51.323778 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:51.323778 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:51.323778 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:51.323778 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:51.323778 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:51.323778 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:51.323778 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:51.323778 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:51.324858 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:51.324858 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:51.324858 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:51.324858 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:51.324858 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:00:51.324858 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:00:51.324858 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:00:51.324858 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:00:51.324858 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:00:51.324858 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:00:51.324858 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 02 19:00:51.324858 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:00:51.324858 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 326/663] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 2 19:00:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:51.324858 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5126d3-470b-4ca5-b1a2-ea6b12eb369e None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:51.346267 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0c6766-2329-4274-8d0a-a6052b744d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:51.346977 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c6766-2329-4274-8d0a-a6052b744d82 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:51.347265 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c6766-2329-4274-8d0a-a6052b744d82 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:51.347547 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c6766-2329-4274-8d0a-a6052b744d82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:51.356997 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:51.426097 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0c6766-2329-4274-8d0a-a6052b744d82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:51.427706 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0c6766-2329-4274-8d0a-a6052b744d82 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:51.450559 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5126d3-470b-4ca5-b1a2-ea6b12eb369e None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:51.493155 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8a25e6-6e2f-420f-aceb-cb326767881b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:51.493881 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a25e6-6e2f-420f-aceb-cb326767881b None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:51.494228 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a25e6-6e2f-420f-aceb-cb326767881b None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:51.494427 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a25e6-6e2f-420f-aceb-cb326767881b None admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:51.495287 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8a25e6-6e2f-420f-aceb-cb326767881b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:51.495603 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8a25e6-6e2f-420f-aceb-cb326767881b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:51.497079 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8a25e6-6e2f-420f-aceb-cb326767881b None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:51.514974 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:51.524393 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 327/664] 10.222.0.33 () {56 vars in 1139 bytes} [Thu Jun 2 19:00:51 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:51.524393 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8a25e6-6e2f-420f-aceb-cb326767881b None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:51.547779 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 338/665] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 2 19:00:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 02 19:00:51.547779 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0c6766-2329-4274-8d0a-a6052b744d82 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:51.639487 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:51.664046 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8a25e6-6e2f-420f-aceb-cb326767881b None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:51.673734 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0c6766-2329-4274-8d0a-a6052b744d82 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:51.704784 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da3c861-461b-4180-b064-e5c765248a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:51.705601 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3c861-461b-4180-b064-e5c765248a38 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:51.705888 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3c861-461b-4180-b064-e5c765248a38 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:51.706200 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3c861-461b-4180-b064-e5c765248a38 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:51.714390 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da3c861-461b-4180-b064-e5c765248a38 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:51.715906 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da3c861-461b-4180-b064-e5c765248a38 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:51.719525 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b00da66-24e3-4009-875d-fddc3b455ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:51.720215 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00da66-24e3-4009-875d-fddc3b455ff6 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:51.720509 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00da66-24e3-4009-875d-fddc3b455ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:51.720785 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00da66-24e3-4009-875d-fddc3b455ff6 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:51.721612 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b00da66-24e3-4009-875d-fddc3b455ff6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:51.721930 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b00da66-24e3-4009-875d-fddc3b455ff6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:51.723370 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b00da66-24e3-4009-875d-fddc3b455ff6 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:51.730883 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-5da3c861-461b-4180-b064-e5c765248a38 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:00:51.730883 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:51.730883 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:51.730883 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:51.730883 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:51.730883 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:51.730883 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:51.730883 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:51.730883 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:51.730883 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:51.730883 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:51.730883 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:51.730883 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 02 19:00:51.730883 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 02 19:00:51.730883 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 02 19:00:51.746940 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 02 19:00:51.746940 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:51.746940 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:51.746940 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:51.746940 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:51.746940 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:51.746940 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:51.746940 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:51.746940 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:51.746940 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:51.746940 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:51.746940 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:51.746940 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:51.746940 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:51.746940 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:51.746940 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:51.746940 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:51.747842 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 02 19:00:51.747842 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 02 19:00:51.747842 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 02 19:00:51.747842 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 02 19:00:51.747842 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 02 19:00:51.747842 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 02 19:00:51.747842 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:00:51.747842 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:00:51.747842 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 328/666] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 2 19:00:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:51.747842 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da3c861-461b-4180-b064-e5c765248a38 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:51.748372 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 339/667] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 2 19:00:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:51.748372 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b00da66-24e3-4009-875d-fddc3b455ff6 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:51.783046 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:51.859699 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b00da66-24e3-4009-875d-fddc3b455ff6 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:51.875144 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da3c861-461b-4180-b064-e5c765248a38 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:51.913970 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d31123-2bf4-48d7-abd8-8179a1edd7d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:51.914829 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d31123-2bf4-48d7-abd8-8179a1edd7d8 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:51.915112 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d31123-2bf4-48d7-abd8-8179a1edd7d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:51.915389 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d31123-2bf4-48d7-abd8-8179a1edd7d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:51.921859 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade88e8b-a0d1-498e-9e1d-b74c634df4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:51.922966 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade88e8b-a0d1-498e-9e1d-b74c634df4a5 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:51.922966 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade88e8b-a0d1-498e-9e1d-b74c634df4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:51.923158 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade88e8b-a0d1-498e-9e1d-b74c634df4a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:51.928955 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d31123-2bf4-48d7-abd8-8179a1edd7d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:51.930985 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d31123-2bf4-48d7-abd8-8179a1edd7d8 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:51.934549 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:51.949870 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-99d31123-2bf4-48d7-abd8-8179a1edd7d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:51.949870 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:51.949870 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:51.949870 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:51.949870 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:51.949870 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:51.949870 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:51.949870 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:51.949870 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:51.949870 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:51.949870 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:51.949870 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:51.949870 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:00:51.949870 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:00:51.949870 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:00:51.951028 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:00:51.951028 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:51.951028 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:51.951028 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:51.951028 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:51.951028 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:51.951028 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:51.951028 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:51.951028 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:51.951028 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:51.951028 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:51.951028 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:51.951028 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:51.951028 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:51.951028 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:51.951028 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:51.951028 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:51.951919 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:00:51.951919 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:00:51.951919 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:51.951919 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:00:51.951919 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 340/668] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:00:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:51.951919 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d31123-2bf4-48d7-abd8-8179a1edd7d8 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:52.020678 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade88e8b-a0d1-498e-9e1d-b74c634df4a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:52.022393 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade88e8b-a0d1-498e-9e1d-b74c634df4a5 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:52.065597 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:52.080019 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d31123-2bf4-48d7-abd8-8179a1edd7d8 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:52.129960 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0e2479-a6c2-4518-8b73-92d01d7ff49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:52.130903 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0e2479-a6c2-4518-8b73-92d01d7ff49c None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:52.131263 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0e2479-a6c2-4518-8b73-92d01d7ff49c None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:52.131619 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0e2479-a6c2-4518-8b73-92d01d7ff49c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:52.146483 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0e2479-a6c2-4518-8b73-92d01d7ff49c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:52.152728 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 329/669] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 2 19:00:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 02 19:00:52.152871 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0e2479-a6c2-4518-8b73-92d01d7ff49c None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:52.152949 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade88e8b-a0d1-498e-9e1d-b74c634df4a5 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:52.184280 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-fa0e2479-a6c2-4518-8b73-92d01d7ff49c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:52.184280 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:52.184280 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:52.184280 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:52.184280 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:52.184280 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:52.184280 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:52.184280 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:52.184280 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:52.184280 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:52.184280 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:52.184280 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:52.184280 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:00:52.184280 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:00:52.184280 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:00:52.185219 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:00:52.185219 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:52.185219 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:52.185219 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:52.185219 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:52.185219 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:52.185219 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:52.185219 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:52.185219 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:52.185219 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:52.185219 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:52.185219 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:52.185219 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:52.185219 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:52.185219 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:52.185219 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:52.185219 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:52.186090 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:00:52.186090 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:00:52.186090 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:52.186090 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:00:52.186090 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 341/670] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:00:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:52.186356 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0e2479-a6c2-4518-8b73-92d01d7ff49c None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:52.209741 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:52.277479 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade88e8b-a0d1-498e-9e1d-b74c634df4a5 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:52.296980 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0e2479-a6c2-4518-8b73-92d01d7ff49c None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:52.312782 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc029f8-ae44-422e-bb1e-914fa7a49c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:52.313450 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc029f8-ae44-422e-bb1e-914fa7a49c2f None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:52.313736 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc029f8-ae44-422e-bb1e-914fa7a49c2f None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:52.314019 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc029f8-ae44-422e-bb1e-914fa7a49c2f None admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:52.314832 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc029f8-ae44-422e-bb1e-914fa7a49c2f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:52.315142 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc029f8-ae44-422e-bb1e-914fa7a49c2f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:52.316722 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc029f8-ae44-422e-bb1e-914fa7a49c2f None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:52.328447 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 330/671] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 2 19:00:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:52.329253 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc029f8-ae44-422e-bb1e-914fa7a49c2f None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:52.330162 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2032962e-57ab-4c76-b7dd-da55154549f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:52.330856 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2032962e-57ab-4c76-b7dd-da55154549f9 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:52.331145 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2032962e-57ab-4c76-b7dd-da55154549f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:52.331421 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2032962e-57ab-4c76-b7dd-da55154549f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:52.332144 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2032962e-57ab-4c76-b7dd-da55154549f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:52.332455 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2032962e-57ab-4c76-b7dd-da55154549f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:52.333853 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2032962e-57ab-4c76-b7dd-da55154549f9 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:52.342775 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:52.346434 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 342/672] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:00:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:52.347332 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2032962e-57ab-4c76-b7dd-da55154549f9 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:52.429029 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:52.433491 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc029f8-ae44-422e-bb1e-914fa7a49c2f None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:52.448916 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2032962e-57ab-4c76-b7dd-da55154549f9 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:52.468335 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d97f819-eacf-4a45-8ca0-0d938848fe26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:52.469125 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d97f819-eacf-4a45-8ca0-0d938848fe26 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:52.469411 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d97f819-eacf-4a45-8ca0-0d938848fe26 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:52.469682 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d97f819-eacf-4a45-8ca0-0d938848fe26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:52.480308 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d97f819-eacf-4a45-8ca0-0d938848fe26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:52.481832 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d97f819-eacf-4a45-8ca0-0d938848fe26 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:52.487652 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd9adb8-7ed6-4693-bed7-985cfbfcd126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:52.488355 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd9adb8-7ed6-4693-bed7-985cfbfcd126 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:52.488639 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd9adb8-7ed6-4693-bed7-985cfbfcd126 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:52.488917 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd9adb8-7ed6-4693-bed7-985cfbfcd126 None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:52.489641 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd9adb8-7ed6-4693-bed7-985cfbfcd126 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:52.489962 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd9adb8-7ed6-4693-bed7-985cfbfcd126 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:52.491520 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd9adb8-7ed6-4693-bed7-985cfbfcd126 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:52.499612 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-3d97f819-eacf-4a45-8ca0-0d938848fe26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:52.499612 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:52.499612 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:52.499612 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:52.499612 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:52.499612 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:52.499612 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:52.499612 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:52.499612 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:52.499612 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:52.499612 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:52.499612 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:52.499612 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:00:52.499612 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:00:52.499612 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:00:52.500539 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:00:52.500539 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:52.500539 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:52.500539 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:52.500539 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:52.500539 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:52.500539 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:52.500539 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:52.500539 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:52.500539 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:52.500539 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:52.500539 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:52.500539 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:52.500539 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:52.500539 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:52.500539 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:52.500539 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:52.501402 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:00:52.501402 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:00:52.501402 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:52.501402 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:00:52.501402 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 331/673] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:00:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:52.502133 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c770cab-0ce2-43b5-8547-579ca80e8639 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:52.502308 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c770cab-0ce2-43b5-8547-579ca80e8639 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:52.502553 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c770cab-0ce2-43b5-8547-579ca80e8639 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:52.508125 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 332/674] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:00:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:00:52.508125 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c770cab-0ce2-43b5-8547-579ca80e8639 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:52.514814 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 343/675] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:00:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:52.514814 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd9adb8-7ed6-4693-bed7-985cfbfcd126 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:52.573281 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:52.620164 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c770cab-0ce2-43b5-8547-579ca80e8639 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:52.627196 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd9adb8-7ed6-4693-bed7-985cfbfcd126 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:52.654951 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd068b7-264e-418f-b2c1-7e6f9dccbf7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:52.655610 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd068b7-264e-418f-b2c1-7e6f9dccbf7a None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:52.655896 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd068b7-264e-418f-b2c1-7e6f9dccbf7a None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:52.656171 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd068b7-264e-418f-b2c1-7e6f9dccbf7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:52.657008 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd068b7-264e-418f-b2c1-7e6f9dccbf7a None admin] RBAC: Adding query filter params (role.id=0e56fba112f94ebba363ffab7f4aa9fe, scope.project.id=8c04020eaebd4647804cf8653f32d9e5, user.id=e07892c0aa21449b94a1cc78e3b8a653) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:52.657321 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd068b7-264e-418f-b2c1-7e6f9dccbf7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:52.658769 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd068b7-264e-418f-b2c1-7e6f9dccbf7a None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:52.661282 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fa2225-30ca-43c9-bf35-7ac840a12c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:52.662118 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fa2225-30ca-43c9-bf35-7ac840a12c26 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:52.662405 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fa2225-30ca-43c9-bf35-7ac840a12c26 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:52.662684 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fa2225-30ca-43c9-bf35-7ac840a12c26 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:52.664773 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 333/676] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 2 19:00:52 2022] GET /identity/v3/role_assignments?user.id=e07892c0aa21449b94a1cc78e3b8a653&scope.project.id=8c04020eaebd4647804cf8653f32d9e5&role.id=0e56fba112f94ebba363ffab7f4aa9fe => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:52.665517 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd068b7-264e-418f-b2c1-7e6f9dccbf7a None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:52.670081 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fa2225-30ca-43c9-bf35-7ac840a12c26 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:52.671619 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fa2225-30ca-43c9-bf35-7ac840a12c26 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:52.688678 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-75fa2225-30ca-43c9-bf35-7ac840a12c26 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:00:52.688678 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:52.688678 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:52.688678 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:52.688678 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:52.688678 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:52.688678 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:52.688678 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:52.688678 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:52.688678 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:52.688678 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:52.688678 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:52.688678 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 02 19:00:52.688678 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 02 19:00:52.688678 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 02 19:00:52.690083 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 02 19:00:52.690083 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:52.690083 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:52.690083 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:52.690083 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:52.690083 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:52.690083 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:52.690083 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:52.690083 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:52.690083 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:52.690083 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:52.690083 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:52.690083 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:52.690083 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:52.690083 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:52.690083 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:52.690083 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:52.691945 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 02 19:00:52.691945 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 02 19:00:52.691945 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 02 19:00:52.691945 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 02 19:00:52.691945 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 02 19:00:52.691945 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 02 19:00:52.691945 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:00:52.691945 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:00:52.691945 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 344/677] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 2 19:00:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:52.691945 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705aadda-574c-4733-9fda-fc1aa40636be None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:52.691945 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705aadda-574c-4733-9fda-fc1aa40636be None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:52.692598 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705aadda-574c-4733-9fda-fc1aa40636be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:52.699578 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:52.712634 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-705aadda-574c-4733-9fda-fc1aa40636be None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:00:52.714268 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-705aadda-574c-4733-9fda-fc1aa40636be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:00:52.792205 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 345/678] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:00:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 19:00:52.793455 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705aadda-574c-4733-9fda-fc1aa40636be None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:52.817370 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd068b7-264e-418f-b2c1-7e6f9dccbf7a None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:52.849331 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:52.859427 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66304735-9dc9-47a4-af94-ae03e74de4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:52.863089 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66304735-9dc9-47a4-af94-ae03e74de4ac None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:52.863089 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66304735-9dc9-47a4-af94-ae03e74de4ac None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:52.863089 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66304735-9dc9-47a4-af94-ae03e74de4ac None admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:52.863089 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66304735-9dc9-47a4-af94-ae03e74de4ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:52.863089 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66304735-9dc9-47a4-af94-ae03e74de4ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:52.863089 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66304735-9dc9-47a4-af94-ae03e74de4ac None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:52.881731 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 334/679] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:00:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:52.881731 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66304735-9dc9-47a4-af94-ae03e74de4ac None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:52.908751 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705aadda-574c-4733-9fda-fc1aa40636be None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:52.950467 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765cdc6d-a1b7-45fd-8bd2-be031e8fa7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:52.951753 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765cdc6d-a1b7-45fd-8bd2-be031e8fa7e8 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:52.951753 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765cdc6d-a1b7-45fd-8bd2-be031e8fa7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:52.951939 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765cdc6d-a1b7-45fd-8bd2-be031e8fa7e8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:52.960130 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765cdc6d-a1b7-45fd-8bd2-be031e8fa7e8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:52.962183 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765cdc6d-a1b7-45fd-8bd2-be031e8fa7e8 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:52.977575 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-765cdc6d-a1b7-45fd-8bd2-be031e8fa7e8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 02 19:00:52.977575 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:52.977575 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:52.977575 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:52.977575 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:52.977575 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:52.977575 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:52.977575 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:52.977575 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:52.977575 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:52.977575 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:52.977575 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:52.977575 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 02 19:00:52.977575 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 02 19:00:52.977575 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 02 19:00:52.977575 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 02 19:00:52.979500 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:52.979500 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:52.979500 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 02 19:00:52.979500 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:00:52.979500 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 02 19:00:52.979500 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:00:52.979500 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:52.979500 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:52.979500 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:52.979500 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:52.979500 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:52.979500 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:52.979500 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:52.979500 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:52.979500 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:52.979500 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:52.979500 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:52.979500 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:52.981192 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:52.981192 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:52.981192 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 02 19:00:52.981192 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 02 19:00:52.981192 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 02 19:00:52.981192 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 02 19:00:52.981192 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 02 19:00:52.981192 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 02 19:00:52.981192 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 02 19:00:52.981192 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:00:52.981192 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 346/680] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 2 19:00:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:52.981192 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765cdc6d-a1b7-45fd-8bd2-be031e8fa7e8 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:52.995418 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:53.035072 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66304735-9dc9-47a4-af94-ae03e74de4ac None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:53.082476 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e0962d-ee48-4781-8553-d9d77334702c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:53.083518 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e0962d-ee48-4781-8553-d9d77334702c None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:53.083893 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e0962d-ee48-4781-8553-d9d77334702c None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:53.084197 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e0962d-ee48-4781-8553-d9d77334702c None admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:53.085256 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e0962d-ee48-4781-8553-d9d77334702c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:53.085569 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e0962d-ee48-4781-8553-d9d77334702c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:53.087651 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e0962d-ee48-4781-8553-d9d77334702c None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:53.105611 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 335/681] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 2 19:00:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:53.105611 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e0962d-ee48-4781-8553-d9d77334702c None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:53.105883 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765cdc6d-a1b7-45fd-8bd2-be031e8fa7e8 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:53.120477 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:53.145247 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e99b17f-4c6b-4133-8b6b-ca369ce7d8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:53.146165 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e99b17f-4c6b-4133-8b6b-ca369ce7d8b3 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:53.146499 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e99b17f-4c6b-4133-8b6b-ca369ce7d8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:53.146795 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e99b17f-4c6b-4133-8b6b-ca369ce7d8b3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:53.158779 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e99b17f-4c6b-4133-8b6b-ca369ce7d8b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:00:53.167187 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-5e99b17f-4c6b-4133-8b6b-ca369ce7d8b3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 02 19:00:53.167187 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:53.167187 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:53.167187 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:53.167187 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:53.167187 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:53.167187 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:53.167187 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:53.167187 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:53.167187 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:53.167187 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:53.167187 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:53.167187 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:00:53.167187 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:00:53.167187 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:00:53.167187 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:00:53.168110 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:00:53.168110 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:00:53.168110 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:00:53.168110 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:00:53.168110 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:53.168110 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:53.168110 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:53.168110 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:53.168110 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:53.168110 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:53.168110 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:53.168110 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:53.168110 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:53.168110 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:53.168110 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:53.168110 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:53.174420 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:53.174420 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:53.174420 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:53.174420 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:53.174420 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:00:53.174420 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:00:53.174420 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:00:53.174420 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:00:53.174420 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:00:53.174420 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:00:53.174420 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 02 19:00:53.174420 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:00:53.174420 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 347/682] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 2 19:00:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:53.174420 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e99b17f-4c6b-4133-8b6b-ca369ce7d8b3 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:53.223558 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e0962d-ee48-4781-8553-d9d77334702c None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:53.260974 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0383ef1-0886-4a21-b782-fba9fff31bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:53.262121 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0383ef1-0886-4a21-b782-fba9fff31bb8 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:53.262646 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0383ef1-0886-4a21-b782-fba9fff31bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:53.263127 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0383ef1-0886-4a21-b782-fba9fff31bb8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:53.267203 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:53.273978 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0383ef1-0886-4a21-b782-fba9fff31bb8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:53.275416 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0383ef1-0886-4a21-b782-fba9fff31bb8 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:53.276266 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e99b17f-4c6b-4133-8b6b-ca369ce7d8b3 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:53.291111 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-b0383ef1-0886-4a21-b782-fba9fff31bb8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:00:53.291111 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:53.291111 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:53.291111 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:53.291111 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:53.291111 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:53.291111 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:53.291111 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:53.291111 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:53.291111 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:53.291111 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:53.291111 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:53.291111 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 02 19:00:53.291111 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 02 19:00:53.291111 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 02 19:00:53.292376 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 02 19:00:53.292376 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:53.292376 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:53.292376 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:53.292376 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:53.292376 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:53.292376 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:53.292376 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:53.292376 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:53.292376 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:53.292376 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:53.292376 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:53.292376 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:53.292376 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:53.292376 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:53.292376 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:53.292376 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:53.293466 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 02 19:00:53.293466 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 02 19:00:53.293466 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 02 19:00:53.293466 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 02 19:00:53.293466 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 02 19:00:53.293466 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 02 19:00:53.293466 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:00:53.293466 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:00:53.293466 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 336/683] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 2 19:00:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:53.293466 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0383ef1-0886-4a21-b782-fba9fff31bb8 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:53.317473 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17ab15f-df84-4ce9-aeb6-e7e4b007b0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:53.318159 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ab15f-df84-4ce9-aeb6-e7e4b007b0c0 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:53.318446 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ab15f-df84-4ce9-aeb6-e7e4b007b0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:53.318723 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ab15f-df84-4ce9-aeb6-e7e4b007b0c0 None admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:53.319597 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17ab15f-df84-4ce9-aeb6-e7e4b007b0c0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:53.319908 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17ab15f-df84-4ce9-aeb6-e7e4b007b0c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:53.321399 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17ab15f-df84-4ce9-aeb6-e7e4b007b0c0 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:53.342013 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 348/684] 10.222.0.33 () {56 vars in 1135 bytes} [Thu Jun 2 19:00:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:53.342892 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17ab15f-df84-4ce9-aeb6-e7e4b007b0c0 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:53.373857 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:53.410522 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0383ef1-0886-4a21-b782-fba9fff31bb8 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:53.446410 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a801510-c4f6-4cd0-b74c-ee6bcea6210c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:53.447111 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a801510-c4f6-4cd0-b74c-ee6bcea6210c None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:53.447399 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a801510-c4f6-4cd0-b74c-ee6bcea6210c None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:53.447850 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a801510-c4f6-4cd0-b74c-ee6bcea6210c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:53.448690 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a801510-c4f6-4cd0-b74c-ee6bcea6210c None admin] RBAC: Adding query filter params (role.id=0e56fba112f94ebba363ffab7f4aa9fe, scope.project.id=8c04020eaebd4647804cf8653f32d9e5, user.id=091860b2577f49c5af61c75803a90c0d) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:53.449003 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a801510-c4f6-4cd0-b74c-ee6bcea6210c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:53.450448 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a801510-c4f6-4cd0-b74c-ee6bcea6210c None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:53.453472 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17ab15f-df84-4ce9-aeb6-e7e4b007b0c0 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:53.456696 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 337/685] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 2 19:00:53 2022] GET /identity/v3/role_assignments?user.id=091860b2577f49c5af61c75803a90c0d&scope.project.id=8c04020eaebd4647804cf8653f32d9e5&role.id=0e56fba112f94ebba363ffab7f4aa9fe => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:53.457741 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a801510-c4f6-4cd0-b74c-ee6bcea6210c None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:53.502114 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44180e42-7c94-4618-bc5a-1db65e4a51ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:53.502860 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44180e42-7c94-4618-bc5a-1db65e4a51ae None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:53.503150 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44180e42-7c94-4618-bc5a-1db65e4a51ae None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:53.503428 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44180e42-7c94-4618-bc5a-1db65e4a51ae None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:53.504597 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44180e42-7c94-4618-bc5a-1db65e4a51ae None admin] RBAC: Adding query filter params (name=admin) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:53.504913 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44180e42-7c94-4618-bc5a-1db65e4a51ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:53.506580 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44180e42-7c94-4618-bc5a-1db65e4a51ae None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:53.521287 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 349/686] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 2 19:00:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:53.521287 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44180e42-7c94-4618-bc5a-1db65e4a51ae None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:53.539121 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:53.607342 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a801510-c4f6-4cd0-b74c-ee6bcea6210c None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:53.643328 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:53.649575 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154bf176-2bc0-4415-bb09-a36607db3adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:53.650282 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154bf176-2bc0-4415-bb09-a36607db3adb None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:53.650707 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154bf176-2bc0-4415-bb09-a36607db3adb None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:53.651184 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154bf176-2bc0-4415-bb09-a36607db3adb None admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:53.652004 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154bf176-2bc0-4415-bb09-a36607db3adb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:53.652322 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154bf176-2bc0-4415-bb09-a36607db3adb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:53.654465 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154bf176-2bc0-4415-bb09-a36607db3adb None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:53.657118 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44180e42-7c94-4618-bc5a-1db65e4a51ae None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:53.672567 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 338/687] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 2 19:00:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:53.673464 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154bf176-2bc0-4415-bb09-a36607db3adb None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:53.713240 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e1e766-ba25-4bfb-9120-bafe047d21b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:53.713240 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e1e766-ba25-4bfb-9120-bafe047d21b7 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:53.713849 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e1e766-ba25-4bfb-9120-bafe047d21b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:53.713849 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e1e766-ba25-4bfb-9120-bafe047d21b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:53.844347 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154bf176-2bc0-4415-bb09-a36607db3adb None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:53.863863 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e1e766-ba25-4bfb-9120-bafe047d21b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:53.865659 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e1e766-ba25-4bfb-9120-bafe047d21b7 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:53.871385 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:53.901333 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccdb9ba-456b-476a-976d-c6eb6613767f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:53.902040 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccdb9ba-456b-476a-976d-c6eb6613767f None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:53.906798 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccdb9ba-456b-476a-976d-c6eb6613767f None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:53.906798 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccdb9ba-456b-476a-976d-c6eb6613767f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:53.929467 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccdb9ba-456b-476a-976d-c6eb6613767f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:53.930985 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccdb9ba-456b-476a-976d-c6eb6613767f None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:53.976083 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:53.982267 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-5ccdb9ba-456b-476a-976d-c6eb6613767f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:53.982267 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:53.982267 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:53.982267 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:53.982267 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:53.982267 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:53.982267 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:53.982267 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:53.982267 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:53.982267 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:53.982267 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:53.982267 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:53.982267 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:00:53.982267 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:00:53.982267 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:00:53.988896 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:00:53.988896 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:53.988896 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:53.988896 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:53.988896 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:53.988896 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:53.988896 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:53.988896 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:53.988896 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:53.988896 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:53.988896 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:53.988896 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:53.988896 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:53.988896 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:53.988896 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:53.988896 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:53.988896 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:53.989766 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:00:53.989766 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:00:53.989766 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:53.989766 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:00:53.989766 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 339/689] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:00:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:53.989766 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccdb9ba-456b-476a-976d-c6eb6613767f None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:53.990374 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 350/689] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 2 19:00:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 02 19:00:53.991616 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e1e766-ba25-4bfb-9120-bafe047d21b7 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:54.142948 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e1e766-ba25-4bfb-9120-bafe047d21b7 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:54.185213 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de238297-5d36-4266-a5ce-11bc06047180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:54.186433 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de238297-5d36-4266-a5ce-11bc06047180 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:54.186951 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de238297-5d36-4266-a5ce-11bc06047180 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:54.187467 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de238297-5d36-4266-a5ce-11bc06047180 None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:54.188412 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de238297-5d36-4266-a5ce-11bc06047180 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:54.188947 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de238297-5d36-4266-a5ce-11bc06047180 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:54.190530 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de238297-5d36-4266-a5ce-11bc06047180 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:54.204196 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 351/690] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:00:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:54.205375 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de238297-5d36-4266-a5ce-11bc06047180 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:54.271190 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:54.321276 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de238297-5d36-4266-a5ce-11bc06047180 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:54.349494 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccdb9ba-456b-476a-976d-c6eb6613767f None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:54.362874 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eb4535-cc66-4c25-82f3-87602a09aa64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:54.364028 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb4535-cc66-4c25-82f3-87602a09aa64 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:54.364537 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb4535-cc66-4c25-82f3-87602a09aa64 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:54.365029 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb4535-cc66-4c25-82f3-87602a09aa64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:54.379673 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eb4535-cc66-4c25-82f3-87602a09aa64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:54.381817 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eb4535-cc66-4c25-82f3-87602a09aa64 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:54.403855 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-56eb4535-cc66-4c25-82f3-87602a09aa64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:54.403855 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:54.403855 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:54.403855 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:54.403855 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:54.403855 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:54.403855 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:54.403855 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:54.403855 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:54.403855 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:54.403855 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:54.403855 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:54.403855 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:00:54.403855 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:00:54.403855 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:00:54.404819 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:00:54.404819 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:54.404819 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:54.404819 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:54.404819 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:54.404819 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:54.404819 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:54.404819 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:54.404819 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:54.404819 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:54.404819 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:54.404819 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:54.404819 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:54.404819 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:54.404819 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:54.404819 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:54.404819 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:54.405687 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:00:54.405687 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:00:54.405687 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:54.405687 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:00:54.408423 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 352/691] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:00:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:54.409470 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eb4535-cc66-4c25-82f3-87602a09aa64 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:54.419159 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b31bf7-9534-453d-9814-3725a3fc6222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:54.419828 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b31bf7-9534-453d-9814-3725a3fc6222 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:54.420108 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b31bf7-9534-453d-9814-3725a3fc6222 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:54.420383 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b31bf7-9534-453d-9814-3725a3fc6222 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:54.421224 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b31bf7-9534-453d-9814-3725a3fc6222 None admin] RBAC: Adding query filter params (role.id=0e56fba112f94ebba363ffab7f4aa9fe, scope.project.id=8c04020eaebd4647804cf8653f32d9e5, user.id=7417a54046e34e899d7b03f2f2314deb) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:54.421537 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b31bf7-9534-453d-9814-3725a3fc6222 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:54.423240 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b31bf7-9534-453d-9814-3725a3fc6222 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:54.435046 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 340/692] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 2 19:00:53 2022] GET /identity/v3/role_assignments?user.id=7417a54046e34e899d7b03f2f2314deb&scope.project.id=8c04020eaebd4647804cf8653f32d9e5&role.id=0e56fba112f94ebba363ffab7f4aa9fe => generated 608 bytes in 443 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:54.436201 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b31bf7-9534-453d-9814-3725a3fc6222 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:54.513398 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:54.544111 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eb4535-cc66-4c25-82f3-87602a09aa64 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:54.546793 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:54.596886 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297dbfef-3ace-4bf0-bfc6-02de59e23d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:54.597798 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297dbfef-3ace-4bf0-bfc6-02de59e23d5d None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:54.598082 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297dbfef-3ace-4bf0-bfc6-02de59e23d5d None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:54.598360 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297dbfef-3ace-4bf0-bfc6-02de59e23d5d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:54.619221 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297dbfef-3ace-4bf0-bfc6-02de59e23d5d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:54.620790 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297dbfef-3ace-4bf0-bfc6-02de59e23d5d None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:54.642691 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-297dbfef-3ace-4bf0-bfc6-02de59e23d5d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 02 19:00:54.642691 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:54.642691 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:54.642691 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:54.642691 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:54.642691 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:54.642691 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:54.642691 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:54.642691 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:54.642691 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:54.642691 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:54.642691 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:54.642691 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 02 19:00:54.642691 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 02 19:00:54.642691 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 02 19:00:54.642691 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 02 19:00:54.643723 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:54.643723 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:54.643723 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 02 19:00:54.643723 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:00:54.643723 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 02 19:00:54.643723 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:00:54.643723 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:54.643723 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:54.643723 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:54.643723 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:54.643723 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:54.643723 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:54.643723 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:54.643723 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:54.643723 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:54.643723 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:54.643723 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:54.643723 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:54.644667 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:54.644667 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:54.644667 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 02 19:00:54.644667 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 02 19:00:54.644667 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 02 19:00:54.644667 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 02 19:00:54.644667 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 02 19:00:54.644667 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 02 19:00:54.644667 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 02 19:00:54.644667 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:00:54.645715 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 353/693] 10.222.0.33 () {56 vars in 1093 bytes} [Thu Jun 2 19:00:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:54.650424 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297dbfef-3ace-4bf0-bfc6-02de59e23d5d None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:54.701941 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b31bf7-9534-453d-9814-3725a3fc6222 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:54.767592 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c64ff60-2607-4f61-af85-6847d83ef747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:54.768998 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c64ff60-2607-4f61-af85-6847d83ef747 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:54.769509 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c64ff60-2607-4f61-af85-6847d83ef747 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:54.769993 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c64ff60-2607-4f61-af85-6847d83ef747 None admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:54.771298 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c64ff60-2607-4f61-af85-6847d83ef747 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:54.772305 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c64ff60-2607-4f61-af85-6847d83ef747 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:54.774270 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c64ff60-2607-4f61-af85-6847d83ef747 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:54.791369 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 341/694] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:00:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:54.797399 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c64ff60-2607-4f61-af85-6847d83ef747 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:54.825045 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297dbfef-3ace-4bf0-bfc6-02de59e23d5d None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:54.867631 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305c3e31-d240-4933-b4f6-0eba8571a73d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:54.868374 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305c3e31-d240-4933-b4f6-0eba8571a73d None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:54.868703 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305c3e31-d240-4933-b4f6-0eba8571a73d None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:54.868930 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305c3e31-d240-4933-b4f6-0eba8571a73d None admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:54.869790 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305c3e31-d240-4933-b4f6-0eba8571a73d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:54.870184 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305c3e31-d240-4933-b4f6-0eba8571a73d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:54.871725 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305c3e31-d240-4933-b4f6-0eba8571a73d None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:54.892384 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 354/695] 10.222.0.33 () {56 vars in 1133 bytes} [Thu Jun 2 19:00:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:54.898025 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305c3e31-d240-4933-b4f6-0eba8571a73d None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:54.910532 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:54.919026 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c64ff60-2607-4f61-af85-6847d83ef747 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:54.927880 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:54.963377 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fefc35d-c78d-4597-8f36-f5a87a777d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:54.964449 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fefc35d-c78d-4597-8f36-f5a87a777d41 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:54.964449 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fefc35d-c78d-4597-8f36-f5a87a777d41 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:54.964756 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fefc35d-c78d-4597-8f36-f5a87a777d41 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:54.973969 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fefc35d-c78d-4597-8f36-f5a87a777d41 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:54.976929 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fefc35d-c78d-4597-8f36-f5a87a777d41 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:54.993743 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-2fefc35d-c78d-4597-8f36-f5a87a777d41 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:00:54.993743 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:54.993743 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:54.993743 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:54.993743 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:54.993743 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:54.993743 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:54.993743 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:54.993743 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:54.993743 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:54.993743 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:54.993743 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:54.993743 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 02 19:00:54.993743 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 02 19:00:54.993743 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 02 19:00:54.994696 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 02 19:00:54.994696 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:54.994696 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:54.994696 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:54.994696 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:54.994696 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:54.994696 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:54.994696 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:54.994696 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:54.994696 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:54.994696 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:54.994696 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:54.994696 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:54.994696 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:54.994696 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:54.994696 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:54.994696 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:54.995653 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 02 19:00:54.995653 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 02 19:00:54.995653 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 02 19:00:54.995653 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 02 19:00:54.995653 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 02 19:00:54.995653 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 02 19:00:54.995653 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:00:54.995653 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:00:54.995653 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 342/696] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 2 19:00:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 02 19:00:55.002932 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fefc35d-c78d-4597-8f36-f5a87a777d41 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:55.041597 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305c3e31-d240-4933-b4f6-0eba8571a73d None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:55.085353 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c21dfeb-c2cb-4674-9e96-c5057cb32f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:55.086583 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c21dfeb-c2cb-4674-9e96-c5057cb32f85 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:55.087219 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c21dfeb-c2cb-4674-9e96-c5057cb32f85 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:55.087527 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c21dfeb-c2cb-4674-9e96-c5057cb32f85 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:55.181242 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c21dfeb-c2cb-4674-9e96-c5057cb32f85 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:55.183171 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c21dfeb-c2cb-4674-9e96-c5057cb32f85 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:55.195801 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fefc35d-c78d-4597-8f36-f5a87a777d41 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:55.227320 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:55.239855 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528054eb-a939-43c5-bc31-976b36bfc455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:55.241200 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528054eb-a939-43c5-bc31-976b36bfc455 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:55.241763 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528054eb-a939-43c5-bc31-976b36bfc455 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:55.242937 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528054eb-a939-43c5-bc31-976b36bfc455 None admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:55.244991 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528054eb-a939-43c5-bc31-976b36bfc455 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:55.245541 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528054eb-a939-43c5-bc31-976b36bfc455 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:55.247357 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528054eb-a939-43c5-bc31-976b36bfc455 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:55.266388 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 343/697] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 2 19:00:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:55.272499 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dff642-8a20-41c4-b13e-31b125c55c27 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:55.272499 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dff642-8a20-41c4-b13e-31b125c55c27 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:55.272499 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dff642-8a20-41c4-b13e-31b125c55c27 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:55.274384 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 344/698] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:00:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:00:55.275074 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dff642-8a20-41c4-b13e-31b125c55c27 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:55.293365 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:55.353999 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 355/699] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 2 19:00:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 02 19:00:55.353999 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b87b453-77d6-45e1-904e-49230f7dae03 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:55.353999 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b87b453-77d6-45e1-904e-49230f7dae03 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:55.353999 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b87b453-77d6-45e1-904e-49230f7dae03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:55.379586 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-5b87b453-77d6-45e1-904e-49230f7dae03 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:00:55.381456 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-5b87b453-77d6-45e1-904e-49230f7dae03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:00:55.401238 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dff642-8a20-41c4-b13e-31b125c55c27 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:55.442301 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca0c715-10e4-4a67-b518-5f2058a2009d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:55.443041 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca0c715-10e4-4a67-b518-5f2058a2009d None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:55.443635 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca0c715-10e4-4a67-b518-5f2058a2009d None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:55.443746 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca0c715-10e4-4a67-b518-5f2058a2009d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:55.444570 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca0c715-10e4-4a67-b518-5f2058a2009d None admin] RBAC: Adding query filter params (role.id=0e56fba112f94ebba363ffab7f4aa9fe, scope.project.id=8c04020eaebd4647804cf8653f32d9e5, user.id=2a59e1d67ff94186a3ebcfcb49938a87) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:55.444936 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca0c715-10e4-4a67-b518-5f2058a2009d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:55.446446 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca0c715-10e4-4a67-b518-5f2058a2009d None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:55.460721 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 345/700] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 2 19:00:55 2022] GET /identity/v3/role_assignments?user.id=2a59e1d67ff94186a3ebcfcb49938a87&scope.project.id=8c04020eaebd4647804cf8653f32d9e5&role.id=0e56fba112f94ebba363ffab7f4aa9fe => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:55.460721 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca0c715-10e4-4a67-b518-5f2058a2009d None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:55.496621 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 356/701] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:00:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 19:00:55.549467 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b87b453-77d6-45e1-904e-49230f7dae03 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:55.593745 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:55.639558 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:55.647130 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca0c715-10e4-4a67-b518-5f2058a2009d None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:55.728723 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76704d85-7347-4873-af3c-53124539b98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:55.730395 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76704d85-7347-4873-af3c-53124539b98b None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:55.730983 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76704d85-7347-4873-af3c-53124539b98b None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:55.731470 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76704d85-7347-4873-af3c-53124539b98b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:55.762065 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76704d85-7347-4873-af3c-53124539b98b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:55.763923 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76704d85-7347-4873-af3c-53124539b98b None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:55.765002 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b87b453-77d6-45e1-904e-49230f7dae03 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:55.791859 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-76704d85-7347-4873-af3c-53124539b98b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:55.791859 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:55.791859 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:55.791859 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:55.791859 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:55.791859 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:55.791859 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:55.791859 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:55.791859 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:55.791859 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:55.791859 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:55.791859 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:55.791859 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:00:55.791859 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:00:55.791859 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:00:55.792865 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:00:55.792865 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:55.792865 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:55.792865 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:55.792865 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:55.792865 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:55.792865 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:55.792865 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:55.792865 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:55.792865 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:55.792865 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:55.792865 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:55.792865 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:55.792865 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:55.792865 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:55.792865 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:55.792865 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:55.793742 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:00:55.793742 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:00:55.793742 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:55.793742 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:00:55.794794 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 346/702] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:00:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:55.804317 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76704d85-7347-4873-af3c-53124539b98b None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:55.817098 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14435ac4-0087-47e7-8ef4-f4772f3b09cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:55.818397 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14435ac4-0087-47e7-8ef4-f4772f3b09cc None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:55.819100 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14435ac4-0087-47e7-8ef4-f4772f3b09cc None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:55.819585 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14435ac4-0087-47e7-8ef4-f4772f3b09cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:55.832834 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14435ac4-0087-47e7-8ef4-f4772f3b09cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:55.835256 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14435ac4-0087-47e7-8ef4-f4772f3b09cc None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:55.856971 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-14435ac4-0087-47e7-8ef4-f4772f3b09cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:55.856971 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:55.856971 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:55.856971 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:55.856971 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:55.856971 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:55.856971 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:55.856971 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:55.856971 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:55.856971 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:55.856971 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:55.856971 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:55.856971 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:00:55.856971 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:00:55.856971 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:00:55.857919 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:00:55.857919 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:55.857919 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:55.857919 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:55.857919 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:55.857919 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:55.857919 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:55.857919 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:55.857919 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:55.857919 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:55.857919 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:55.857919 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:55.857919 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:55.857919 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:55.857919 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:55.857919 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:55.857919 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:55.859009 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:00:55.859009 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:00:55.859009 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:55.859009 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:00:55.861531 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 357/703] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:00:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 02 19:00:55.867930 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14435ac4-0087-47e7-8ef4-f4772f3b09cc None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:55.927230 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76704d85-7347-4873-af3c-53124539b98b None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:55.959828 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:55.966695 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c09cc3f-2461-485e-9598-88ec519419ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:55.971902 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c09cc3f-2461-485e-9598-88ec519419ea None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:55.971902 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c09cc3f-2461-485e-9598-88ec519419ea None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:55.971902 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c09cc3f-2461-485e-9598-88ec519419ea None admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:55.971902 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c09cc3f-2461-485e-9598-88ec519419ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:55.971902 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c09cc3f-2461-485e-9598-88ec519419ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:55.971902 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c09cc3f-2461-485e-9598-88ec519419ea None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:55.984958 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 347/704] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:00:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:00:55.990412 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c09cc3f-2461-485e-9598-88ec519419ea None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:55.995579 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14435ac4-0087-47e7-8ef4-f4772f3b09cc None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:55.998042 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:56.040840 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee270c12-92f5-4998-b7ff-3ec4e56b329c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:56.042438 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee270c12-92f5-4998-b7ff-3ec4e56b329c None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:56.043000 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee270c12-92f5-4998-b7ff-3ec4e56b329c None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:56.043508 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee270c12-92f5-4998-b7ff-3ec4e56b329c None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:56.044455 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee270c12-92f5-4998-b7ff-3ec4e56b329c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:56.044966 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee270c12-92f5-4998-b7ff-3ec4e56b329c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:56.046661 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee270c12-92f5-4998-b7ff-3ec4e56b329c None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:56.063719 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 358/705] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:00:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:00:56.069909 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee270c12-92f5-4998-b7ff-3ec4e56b329c None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:56.125989 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c09cc3f-2461-485e-9598-88ec519419ea None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:56.185968 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ea70f6-c758-473f-bc32-2aad80b6a488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:56.187105 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ea70f6-c758-473f-bc32-2aad80b6a488 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:56.187400 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ea70f6-c758-473f-bc32-2aad80b6a488 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:56.187679 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ea70f6-c758-473f-bc32-2aad80b6a488 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:56.198194 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ea70f6-c758-473f-bc32-2aad80b6a488 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:56.200194 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ea70f6-c758-473f-bc32-2aad80b6a488 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:56.209491 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee270c12-92f5-4998-b7ff-3ec4e56b329c None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:56.217186 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-93ea70f6-c758-473f-bc32-2aad80b6a488 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:00:56.217186 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:56.217186 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:56.217186 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:56.217186 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:56.217186 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:56.217186 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:56.217186 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:56.217186 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:56.217186 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:56.217186 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:56.217186 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:56.217186 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 02 19:00:56.217186 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 02 19:00:56.217186 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 02 19:00:56.218112 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 02 19:00:56.218112 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:56.218112 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:56.218112 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:56.218112 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:56.218112 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:56.218112 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:56.218112 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:56.218112 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:56.218112 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:56.218112 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:56.218112 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:56.218112 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:56.218112 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:56.218112 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:56.218112 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:56.218112 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:56.219244 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 02 19:00:56.219244 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 02 19:00:56.219244 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 02 19:00:56.219244 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 02 19:00:56.219244 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 02 19:00:56.219244 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 02 19:00:56.219244 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:00:56.219244 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:00:56.219244 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 348/706] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 2 19:00:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 02 19:00:56.225198 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e7fa5a-d5a3-4c88-b191-1460372d02cf None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:56.225552 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e7fa5a-d5a3-4c88-b191-1460372d02cf None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:56.225926 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e7fa5a-d5a3-4c88-b191-1460372d02cf None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:56.228286 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 349/707] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:00:56 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:00:56.233411 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e7fa5a-d5a3-4c88-b191-1460372d02cf None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:56.250257 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de78b3f1-bf07-46a0-8338-41863cafb18a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:56.251010 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de78b3f1-bf07-46a0-8338-41863cafb18a None admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:56.251587 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de78b3f1-bf07-46a0-8338-41863cafb18a None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:56.251880 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de78b3f1-bf07-46a0-8338-41863cafb18a None admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:56.252612 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de78b3f1-bf07-46a0-8338-41863cafb18a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:56.255254 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de78b3f1-bf07-46a0-8338-41863cafb18a None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:56.298409 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:56.338780 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 359/708] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 2 19:00:56 2022] POST /identity/v3/users => generated 254 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:00:56.341389 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb85446-0e66-4f71-b8ba-92f9160c31f7 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:56.342041 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb85446-0e66-4f71-b8ba-92f9160c31f7 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:56.342792 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb85446-0e66-4f71-b8ba-92f9160c31f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:56.364311 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e7fa5a-d5a3-4c88-b191-1460372d02cf None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:56.370351 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-7bb85446-0e66-4f71-b8ba-92f9160c31f7 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:00:56.372049 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-7bb85446-0e66-4f71-b8ba-92f9160c31f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:00:56.399189 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:56.447591 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81665a52-269e-42e5-a4f0-9d00cbbfacf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:56.449776 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81665a52-269e-42e5-a4f0-9d00cbbfacf9 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:56.450447 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81665a52-269e-42e5-a4f0-9d00cbbfacf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:56.454433 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81665a52-269e-42e5-a4f0-9d00cbbfacf9 None admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:56.455687 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81665a52-269e-42e5-a4f0-9d00cbbfacf9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:56.456240 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81665a52-269e-42e5-a4f0-9d00cbbfacf9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:56.457968 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81665a52-269e-42e5-a4f0-9d00cbbfacf9 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:56.484593 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 350/709] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 2 19:00:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:56.493444 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81665a52-269e-42e5-a4f0-9d00cbbfacf9 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:56.511861 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 360/710] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:00:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 19:00:56.537574 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb85446-0e66-4f71-b8ba-92f9160c31f7 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:56.665251 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81665a52-269e-42e5-a4f0-9d00cbbfacf9 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:56.678772 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:56.709667 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2988e9f3-caa9-4f65-bc2e-98de43579645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:56.710336 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2988e9f3-caa9-4f65-bc2e-98de43579645 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:56.710623 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2988e9f3-caa9-4f65-bc2e-98de43579645 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:56.710961 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2988e9f3-caa9-4f65-bc2e-98de43579645 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:56.711808 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2988e9f3-caa9-4f65-bc2e-98de43579645 None admin] RBAC: Adding query filter params (role.id=f13fdd0847924014abf1e95594195e78, scope.project.id=8c04020eaebd4647804cf8653f32d9e5, user.id=70664d5cbdee47ee89c120761939cc75) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:56.712128 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2988e9f3-caa9-4f65-bc2e-98de43579645 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:56.713589 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2988e9f3-caa9-4f65-bc2e-98de43579645 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:56.717323 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb85446-0e66-4f71-b8ba-92f9160c31f7 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:56.720184 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 351/711] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 2 19:00:56 2022] GET /identity/v3/role_assignments?user.id=70664d5cbdee47ee89c120761939cc75&scope.project.id=8c04020eaebd4647804cf8653f32d9e5&role.id=f13fdd0847924014abf1e95594195e78 => generated 259 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:56.769400 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8424a009-956d-4a3c-867a-9f68bdcc6c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:56.770276 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8424a009-956d-4a3c-867a-9f68bdcc6c45 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:56.770562 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8424a009-956d-4a3c-867a-9f68bdcc6c45 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:56.770881 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8424a009-956d-4a3c-867a-9f68bdcc6c45 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:56.771575 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8424a009-956d-4a3c-867a-9f68bdcc6c45 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:56.773054 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8424a009-956d-4a3c-867a-9f68bdcc6c45 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:56.790454 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-8424a009-956d-4a3c-867a-9f68bdcc6c45 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 02 19:00:56.790454 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:56.790454 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 02 19:00:56.790454 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 02 19:00:56.790454 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 02 19:00:56.790454 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 02 19:00:56.790454 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 02 19:00:56.790454 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 02 19:00:56.790454 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 02 19:00:56.790454 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:00:56.790454 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 02 19:00:56.790454 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:00:56.790454 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:56.790454 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:56.790454 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:56.790454 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:56.791694 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:56.791694 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:56.791694 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:56.791694 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:56.791694 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:56.791694 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:56.791694 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:56.791694 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 02 19:00:56.791694 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 02 19:00:56.791694 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 02 19:00:56.791694 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 02 19:00:56.791694 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:56.791694 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:56.791694 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:56.791694 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:56.791694 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:56.792722 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:56.792722 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:56.792722 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:56.792722 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:56.792722 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:56.792722 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:56.792722 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:56.792722 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:56.792722 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:56.792722 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:56.792722 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:56.792722 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 02 19:00:56.792722 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 02 19:00:56.792722 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 02 19:00:56.792722 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:00:56.792722 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 361/712] 10.222.0.33 () {56 vars in 1129 bytes} [Thu Jun 2 19:00:56 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:56.802077 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2988e9f3-caa9-4f65-bc2e-98de43579645 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:56.930489 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2988e9f3-caa9-4f65-bc2e-98de43579645 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:56.987746 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8d6f94-0071-43f0-9325-2ea8c390117c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:56.988467 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8d6f94-0071-43f0-9325-2ea8c390117c None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:56.988758 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8d6f94-0071-43f0-9325-2ea8c390117c None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:56.989057 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8d6f94-0071-43f0-9325-2ea8c390117c None admin] PATH_INFO: `/v3/services` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:56.989791 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8d6f94-0071-43f0-9325-2ea8c390117c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:56.990104 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8d6f94-0071-43f0-9325-2ea8c390117c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:56.991701 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8d6f94-0071-43f0-9325-2ea8c390117c None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:56.997979 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 352/713] 10.222.0.33 () {56 vars in 1124 bytes} [Thu Jun 2 19:00:56 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:57.005214 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8424a009-956d-4a3c-867a-9f68bdcc6c45 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:57.029303 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:57.165785 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8424a009-956d-4a3c-867a-9f68bdcc6c45 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:57.217435 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeb821b-e68d-4e46-b8b2-a9c8099e2636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:57.218034 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeb821b-e68d-4e46-b8b2-a9c8099e2636 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:57.218322 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeb821b-e68d-4e46-b8b2-a9c8099e2636 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:57.218597 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeb821b-e68d-4e46-b8b2-a9c8099e2636 None admin] PATH_INFO: `/v3/services` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:57.219375 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeb821b-e68d-4e46-b8b2-a9c8099e2636 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:57.219687 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeb821b-e68d-4e46-b8b2-a9c8099e2636 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:57.221073 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeb821b-e68d-4e46-b8b2-a9c8099e2636 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:57.232652 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 362/714] 10.222.0.33 () {56 vars in 1124 bytes} [Thu Jun 2 19:00:57 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:57.477253 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e8a40-20cf-47c3-b6f3-f8e2785efa53 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:57.477634 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e8a40-20cf-47c3-b6f3-f8e2785efa53 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:57.477847 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e8a40-20cf-47c3-b6f3-f8e2785efa53 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:57.479588 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 353/715] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:00:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:00:57.485458 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64dba93-63f2-4986-b7e7-21d04205ff81 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:57.485755 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64dba93-63f2-4986-b7e7-21d04205ff81 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:57.486019 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64dba93-63f2-4986-b7e7-21d04205ff81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:57.512658 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-b64dba93-63f2-4986-b7e7-21d04205ff81 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:00:57.513081 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-b64dba93-63f2-4986-b7e7-21d04205ff81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:00:57.600538 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 363/716] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:00:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 19:00:57.623883 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91e8a40-20cf-47c3-b6f3-f8e2785efa53 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:57.682635 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:57.745211 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91e8a40-20cf-47c3-b6f3-f8e2785efa53 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:57.783911 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec1bb59-7d7c-4a84-9745-6601dcad1917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:57.784714 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec1bb59-7d7c-4a84-9745-6601dcad1917 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:57.784996 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec1bb59-7d7c-4a84-9745-6601dcad1917 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:57.785267 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec1bb59-7d7c-4a84-9745-6601dcad1917 None admin] PATH_INFO: `/v3/services/network` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:57.785916 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec1bb59-7d7c-4a84-9745-6601dcad1917 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:57.787529 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec1bb59-7d7c-4a84-9745-6601dcad1917 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:57.802575 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-5ec1bb59-7d7c-4a84-9745-6601dcad1917 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 02 19:00:57.802575 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:57.802575 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 02 19:00:57.802575 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 02 19:00:57.802575 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 02 19:00:57.802575 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 02 19:00:57.802575 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 02 19:00:57.802575 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 02 19:00:57.802575 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 02 19:00:57.802575 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:00:57.802575 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 02 19:00:57.802575 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:00:57.802575 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:57.802575 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:57.802575 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:57.802575 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:57.802575 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:57.804038 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:57.804038 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:57.804038 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:57.804038 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:57.804038 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:57.804038 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:57.804038 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 02 19:00:57.804038 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 02 19:00:57.804038 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 02 19:00:57.804038 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 02 19:00:57.804038 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:57.804038 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:57.804038 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:57.804038 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:57.804038 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:57.804038 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:57.804891 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:57.804891 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:57.804891 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:57.804891 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:57.804891 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:57.804891 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:57.804891 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:57.804891 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:57.804891 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:57.804891 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:57.804891 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 02 19:00:57.804891 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 02 19:00:57.804891 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 02 19:00:57.804891 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:00:57.804891 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 354/717] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 2 19:00:57 2022] GET /identity/v3/services/network => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:57.809910 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64dba93-63f2-4986-b7e7-21d04205ff81 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:57.823383 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:57.942939 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64dba93-63f2-4986-b7e7-21d04205ff81 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:57.994072 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a43385-88f6-4d62-899a-c5ef0d27c9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:57.994789 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a43385-88f6-4d62-899a-c5ef0d27c9e2 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:57.995087 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a43385-88f6-4d62-899a-c5ef0d27c9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:57.995419 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a43385-88f6-4d62-899a-c5ef0d27c9e2 None admin] PATH_INFO: `/v3/services` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:57.996107 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a43385-88f6-4d62-899a-c5ef0d27c9e2 None admin] RBAC: Adding query filter params (name=network) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:57.996420 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a43385-88f6-4d62-899a-c5ef0d27c9e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:57.997868 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a43385-88f6-4d62-899a-c5ef0d27c9e2 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:58.003669 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 364/718] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Jun 2 19:00:57 2022] GET /identity/v3/services?name=network => generated 124 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:58.010026 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec1bb59-7d7c-4a84-9745-6601dcad1917 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:58.150832 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec1bb59-7d7c-4a84-9745-6601dcad1917 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:58.163186 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:58.201880 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dbbf1f-1caf-492b-97f2-fd20b409e7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:58.202580 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dbbf1f-1caf-492b-97f2-fd20b409e7a9 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:58.202932 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dbbf1f-1caf-492b-97f2-fd20b409e7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:58.203215 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dbbf1f-1caf-492b-97f2-fd20b409e7a9 None admin] PATH_INFO: `/v3/services` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:58.204009 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dbbf1f-1caf-492b-97f2-fd20b409e7a9 None admin] RBAC: Adding query filter params (type=network) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:58.204325 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dbbf1f-1caf-492b-97f2-fd20b409e7a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:58.205795 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dbbf1f-1caf-492b-97f2-fd20b409e7a9 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:58.217295 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 355/719] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Jun 2 19:00:58 2022] GET /identity/v3/services?type=network => generated 124 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:58.587784 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6cb9eb-3c28-476a-b2b1-f27f9203d1b3 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:58.588820 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6cb9eb-3c28-476a-b2b1-f27f9203d1b3 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:58.589340 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6cb9eb-3c28-476a-b2b1-f27f9203d1b3 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:58.591289 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 365/720] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:00:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:00:58.596557 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d560ecd0-2202-4c91-affc-adc842ec9f08 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:58.597116 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d560ecd0-2202-4c91-affc-adc842ec9f08 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:58.597616 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d560ecd0-2202-4c91-affc-adc842ec9f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:58.617877 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-d560ecd0-2202-4c91-affc-adc842ec9f08 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:00:58.619461 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-d560ecd0-2202-4c91-affc-adc842ec9f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:00:58.698448 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 356/721] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:00:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 19:00:58.734713 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6cb9eb-3c28-476a-b2b1-f27f9203d1b3 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:58.835304 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:58.884199 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6cb9eb-3c28-476a-b2b1-f27f9203d1b3 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:58.939729 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d39a740-e57d-4298-bfa3-542e5fee7496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:58.940604 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d39a740-e57d-4298-bfa3-542e5fee7496 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:58.940888 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d39a740-e57d-4298-bfa3-542e5fee7496 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:58.941169 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d39a740-e57d-4298-bfa3-542e5fee7496 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:58.955540 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d39a740-e57d-4298-bfa3-542e5fee7496 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:00:58.964156 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-8d39a740-e57d-4298-bfa3-542e5fee7496 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 02 19:00:58.964156 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:58.964156 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:58.964156 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:58.964156 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:58.964156 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:58.964156 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:58.964156 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:58.964156 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:58.964156 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:58.964156 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:58.964156 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:58.964156 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:00:58.964156 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:00:58.964156 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:00:58.964156 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:00:58.965179 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:00:58.965179 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:00:58.965179 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:00:58.965179 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:00:58.965179 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:58.965179 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:58.965179 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:58.965179 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:58.965179 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:58.965179 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:58.965179 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:58.965179 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:58.965179 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:58.965179 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:58.965179 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:58.965179 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:58.966012 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:58.966012 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:58.966012 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:58.966012 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:58.966012 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:00:58.966012 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:00:58.966012 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:00:58.966012 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:00:58.966012 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:00:58.966012 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:00:58.966012 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 02 19:00:58.966012 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:00:58.968990 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 366/722] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 2 19:00:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:58.975309 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d560ecd0-2202-4c91-affc-adc842ec9f08 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:59.079263 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:59.091146 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d560ecd0-2202-4c91-affc-adc842ec9f08 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:59.146378 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbe88e2-ed3c-4682-b125-7f43a2cb9fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:59.147014 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe88e2-ed3c-4682-b125-7f43a2cb9fd7 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:59.147309 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe88e2-ed3c-4682-b125-7f43a2cb9fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:59.147595 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe88e2-ed3c-4682-b125-7f43a2cb9fd7 None admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:59.148390 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbe88e2-ed3c-4682-b125-7f43a2cb9fd7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:59.149248 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbe88e2-ed3c-4682-b125-7f43a2cb9fd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:59.150627 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbe88e2-ed3c-4682-b125-7f43a2cb9fd7 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:59.165522 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 357/723] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 2 19:00:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:59.171138 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d39a740-e57d-4298-bfa3-542e5fee7496 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:59.334384 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d39a740-e57d-4298-bfa3-542e5fee7496 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:59.375616 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:59.388432 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25c732a-7ca9-4319-b315-ab21956b677c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:59.389258 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c732a-7ca9-4319-b315-ab21956b677c None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:59.389542 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c732a-7ca9-4319-b315-ab21956b677c None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:59.389825 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c732a-7ca9-4319-b315-ab21956b677c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:59.402286 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25c732a-7ca9-4319-b315-ab21956b677c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:59.407795 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25c732a-7ca9-4319-b315-ab21956b677c None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:59.431043 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-a25c732a-7ca9-4319-b315-ab21956b677c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:59.431043 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:59.431043 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:59.431043 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:59.431043 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:59.431043 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:59.431043 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:59.431043 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:59.431043 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:59.431043 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:59.431043 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:59.431043 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:59.431043 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:00:59.431043 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:00:59.431043 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:00:59.432017 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:00:59.432017 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:59.432017 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:59.432017 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:59.432017 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:59.432017 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:59.432017 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:59.432017 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:59.432017 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:59.432017 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:59.432017 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:59.432017 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:59.432017 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:59.432017 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:59.432017 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:59.432017 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:59.432017 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:59.433005 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:00:59.433005 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:00:59.433005 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:00:59.433005 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:00:59.433005 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 367/724] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:00:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:59.437981 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbe88e2-ed3c-4682-b125-7f43a2cb9fd7 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:59.597906 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbe88e2-ed3c-4682-b125-7f43a2cb9fd7 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:59.637301 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233eb25-de34-4fec-9126-f7c033d5c5ae None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:59.638291 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233eb25-de34-4fec-9126-f7c033d5c5ae None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:59.638801 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233eb25-de34-4fec-9126-f7c033d5c5ae None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:59.644544 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 368/725] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:00:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 02 19:00:59.646396 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ec9676-54f9-4925-a0c7-daee5b526e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:59.646837 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e032a9b9-5004-4732-9365-7d8ca2c54086 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:59.647105 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec9676-54f9-4925-a0c7-daee5b526e20 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:59.647396 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec9676-54f9-4925-a0c7-daee5b526e20 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:59.647676 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec9676-54f9-4925-a0c7-daee5b526e20 None admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:59.647928 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e032a9b9-5004-4732-9365-7d8ca2c54086 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:59.648468 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ec9676-54f9-4925-a0c7-daee5b526e20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:00:59.649197 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e032a9b9-5004-4732-9365-7d8ca2c54086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:59.649294 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ec9676-54f9-4925-a0c7-daee5b526e20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:59.650687 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ec9676-54f9-4925-a0c7-daee5b526e20 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:59.667482 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-e032a9b9-5004-4732-9365-7d8ca2c54086 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:00:59.669014 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-e032a9b9-5004-4732-9365-7d8ca2c54086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:00:59.673019 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 358/726] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:00:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:00:59.680501 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ec9676-54f9-4925-a0c7-daee5b526e20 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:59.723301 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:00:59.813349 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 369/727] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:00:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 02 19:00:59.828108 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ec9676-54f9-4925-a0c7-daee5b526e20 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:00:59.854320 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e032a9b9-5004-4732-9365-7d8ca2c54086 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:00:59.896168 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8987f675-c411-45c4-9b30-50f910baad88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:00:59.897029 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8987f675-c411-45c4-9b30-50f910baad88 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:00:59.897318 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8987f675-c411-45c4-9b30-50f910baad88 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:00:59.897597 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8987f675-c411-45c4-9b30-50f910baad88 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:00:59.908805 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8987f675-c411-45c4-9b30-50f910baad88 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:00:59.910304 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8987f675-c411-45c4-9b30-50f910baad88 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:00:59.937185 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-8987f675-c411-45c4-9b30-50f910baad88 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 02 19:00:59.937185 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:00:59.937185 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:00:59.937185 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:00:59.937185 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:00:59.937185 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:00:59.937185 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:00:59.937185 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:00:59.937185 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:00:59.937185 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:00:59.937185 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:00:59.937185 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:00:59.937185 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 02 19:00:59.937185 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 02 19:00:59.937185 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 02 19:00:59.937185 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 02 19:00:59.939492 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:00:59.939492 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:00:59.939492 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 02 19:00:59.939492 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:00:59.939492 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 02 19:00:59.939492 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:00:59.939492 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:00:59.939492 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:00:59.939492 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:00:59.939492 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:00:59.939492 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:00:59.939492 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:00:59.939492 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:00:59.939492 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:00:59.939492 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:00:59.939492 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:00:59.939492 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:00:59.939492 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:00:59.941731 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:00:59.941731 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:00:59.941731 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 02 19:00:59.941731 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 02 19:00:59.941731 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 02 19:00:59.941731 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 02 19:00:59.941731 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 02 19:00:59.941731 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 02 19:00:59.941731 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 02 19:00:59.941731 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:00:59.941731 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 359/728] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 2 19:00:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:00:59.951856 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8987f675-c411-45c4-9b30-50f910baad88 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:00.013435 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e032a9b9-5004-4732-9365-7d8ca2c54086 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:00.043460 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:00.045094 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:00.053166 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6689d4-5928-43b7-823a-1f69311576b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:00.057070 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6689d4-5928-43b7-823a-1f69311576b7 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:00.057070 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6689d4-5928-43b7-823a-1f69311576b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:00.057070 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6689d4-5928-43b7-823a-1f69311576b7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:00.066597 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8987f675-c411-45c4-9b30-50f910baad88 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:00.067023 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6689d4-5928-43b7-823a-1f69311576b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:01:00.075754 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-cd6689d4-5928-43b7-823a-1f69311576b7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 02 19:01:00.075754 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:00.075754 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:00.075754 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:00.075754 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:00.075754 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:00.075754 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:00.075754 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:00.075754 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:00.075754 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:00.075754 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:00.075754 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:00.075754 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:01:00.075754 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:01:00.075754 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:01:00.075754 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:01:00.076734 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:01:00.076734 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:01:00.076734 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:01:00.076734 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:01:00.076734 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:00.076734 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:00.076734 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:00.076734 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:00.076734 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:00.076734 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:00.076734 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:00.076734 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:00.076734 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:00.076734 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:00.076734 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:00.076734 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:00.077560 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:00.077560 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:00.077560 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:00.077560 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:00.077560 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:01:00.077560 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:01:00.077560 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:01:00.077560 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:01:00.077560 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:01:00.077560 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:01:00.077560 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 02 19:01:00.077560 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:00.084862 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 370/729] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 2 19:00:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 02 19:01:00.086682 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4888ff3a-0205-4802-a5a8-88e14c54929e None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:00.087217 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4888ff3a-0205-4802-a5a8-88e14c54929e None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:00.087679 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4888ff3a-0205-4802-a5a8-88e14c54929e None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:00.093535 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 371/730] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:01:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:01:00.094695 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4888ff3a-0205-4802-a5a8-88e14c54929e None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:00.105430 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6ad224-2694-4444-987d-aebf9f49edcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:00.105676 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ad224-2694-4444-987d-aebf9f49edcf None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:00.105903 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ad224-2694-4444-987d-aebf9f49edcf None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:00.106188 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ad224-2694-4444-987d-aebf9f49edcf None admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:00.107103 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6ad224-2694-4444-987d-aebf9f49edcf None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:00.107419 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6ad224-2694-4444-987d-aebf9f49edcf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:00.109078 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6ad224-2694-4444-987d-aebf9f49edcf None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:00.129012 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 360/731] 10.222.0.33 () {56 vars in 1135 bytes} [Thu Jun 2 19:00:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:00.130723 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48787715-7028-4317-839e-9521d651b61e None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:00.131068 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48787715-7028-4317-839e-9521d651b61e None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:00.131349 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48787715-7028-4317-839e-9521d651b61e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:00.150182 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-48787715-7028-4317-839e-9521d651b61e None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:01:00.151431 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-48787715-7028-4317-839e-9521d651b61e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:01:00.218628 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4888ff3a-0205-4802-a5a8-88e14c54929e None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:00.232749 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 361/732] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:01:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 19:01:00.233913 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48787715-7028-4317-839e-9521d651b61e None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:00.278377 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66817ab8-bbd9-4dbc-9e2d-611b7c770166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:00.279085 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66817ab8-bbd9-4dbc-9e2d-611b7c770166 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:00.279370 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66817ab8-bbd9-4dbc-9e2d-611b7c770166 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:00.279646 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66817ab8-bbd9-4dbc-9e2d-611b7c770166 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:00.280572 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66817ab8-bbd9-4dbc-9e2d-611b7c770166 None admin] RBAC: Adding query filter params (name=service) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:00.280684 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66817ab8-bbd9-4dbc-9e2d-611b7c770166 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:00.282068 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66817ab8-bbd9-4dbc-9e2d-611b7c770166 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:00.295950 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 372/733] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 2 19:01:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:00.296755 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66817ab8-bbd9-4dbc-9e2d-611b7c770166 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:00.310395 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:00.330891 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:00.349043 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48787715-7028-4317-839e-9521d651b61e None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:00.387956 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbd3be1-4f61-443a-8437-c9e8977eefc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:00.388998 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbd3be1-4f61-443a-8437-c9e8977eefc8 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:00.389499 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbd3be1-4f61-443a-8437-c9e8977eefc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:00.390045 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbd3be1-4f61-443a-8437-c9e8977eefc8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:00.398126 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66817ab8-bbd9-4dbc-9e2d-611b7c770166 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:00.433500 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32907919-0f17-4f65-bb08-6d3df484dbdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:00.434465 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32907919-0f17-4f65-bb08-6d3df484dbdd None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:00.434871 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32907919-0f17-4f65-bb08-6d3df484dbdd None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:00.435246 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32907919-0f17-4f65-bb08-6d3df484dbdd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:00.446657 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32907919-0f17-4f65-bb08-6d3df484dbdd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:01:00.454481 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-32907919-0f17-4f65-bb08-6d3df484dbdd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 02 19:01:00.454481 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:00.454481 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:00.454481 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:00.454481 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:00.454481 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:00.454481 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:00.454481 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:00.454481 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:00.454481 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:00.454481 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:00.454481 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:00.454481 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:01:00.454481 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:01:00.454481 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:01:00.454481 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:01:00.455671 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:01:00.455671 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:01:00.455671 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:01:00.455671 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:01:00.455671 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:00.455671 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:00.455671 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:00.455671 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:00.455671 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:00.455671 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:00.455671 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:00.455671 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:00.455671 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:00.455671 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:00.455671 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:00.455671 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:00.456701 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:00.456701 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:00.456701 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:00.456701 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:00.456701 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:01:00.456701 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:01:00.456701 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:01:00.456701 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:01:00.456701 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:01:00.456701 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:01:00.456701 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 02 19:01:00.456701 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:00.462866 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 373/734] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 2 19:01:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:00.463895 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32907919-0f17-4f65-bb08-6d3df484dbdd None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:00.473053 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbd3be1-4f61-443a-8437-c9e8977eefc8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:00.474881 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbd3be1-4f61-443a-8437-c9e8977eefc8 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:00.547821 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:00.563562 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:00.569747 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32907919-0f17-4f65-bb08-6d3df484dbdd None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:00.599554 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 362/735] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 2 19:01:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 02 19:01:00.600624 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbd3be1-4f61-443a-8437-c9e8977eefc8 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:00.603290 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e4484b-1cbb-4b3b-b496-583d0153a76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:00.604084 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e4484b-1cbb-4b3b-b496-583d0153a76a None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:00.604654 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e4484b-1cbb-4b3b-b496-583d0153a76a None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:00.604765 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e4484b-1cbb-4b3b-b496-583d0153a76a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:00.617084 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e4484b-1cbb-4b3b-b496-583d0153a76a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:00.619228 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e4484b-1cbb-4b3b-b496-583d0153a76a None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:00.637599 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-42e4484b-1cbb-4b3b-b496-583d0153a76a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:01:00.637599 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:00.637599 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:00.637599 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:00.637599 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:00.637599 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:00.637599 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:00.637599 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:00.637599 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:00.637599 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:00.637599 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:00.637599 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:00.637599 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:01:00.637599 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:01:00.637599 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:01:00.638989 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:01:00.638989 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:00.638989 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:00.638989 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:00.638989 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:00.638989 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:00.638989 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:00.638989 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:00.638989 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:00.638989 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:00.638989 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:00.638989 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:00.638989 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:00.638989 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:00.638989 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:00.638989 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:00.638989 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:00.639886 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:01:00.639886 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:01:00.639886 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:01:00.639886 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:00.639886 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 374/736] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:01:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:00.640188 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecb9882-d8df-4683-9f33-ac6707901af2 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:00.640552 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecb9882-d8df-4683-9f33-ac6707901af2 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:00.640650 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecb9882-d8df-4683-9f33-ac6707901af2 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:00.642038 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 375/737] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:01:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:01:00.642637 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecb9882-d8df-4683-9f33-ac6707901af2 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:00.719979 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbd3be1-4f61-443a-8437-c9e8977eefc8 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:00.755934 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6905a38-df3e-4c2d-adc8-9331b4536278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:00.756492 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6905a38-df3e-4c2d-adc8-9331b4536278 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:00.756765 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6905a38-df3e-4c2d-adc8-9331b4536278 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:00.757072 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6905a38-df3e-4c2d-adc8-9331b4536278 None admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:00.757866 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6905a38-df3e-4c2d-adc8-9331b4536278 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:00.758175 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6905a38-df3e-4c2d-adc8-9331b4536278 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:00.759598 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6905a38-df3e-4c2d-adc8-9331b4536278 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:00.765331 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecb9882-d8df-4683-9f33-ac6707901af2 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:00.772514 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 363/738] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 2 19:01:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:00.774010 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6905a38-df3e-4c2d-adc8-9331b4536278 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:00.802929 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:00.804560 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7ba458-21d4-4805-8c1f-c67a6ab0dab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:00.805623 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7ba458-21d4-4805-8c1f-c67a6ab0dab6 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:00.806097 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7ba458-21d4-4805-8c1f-c67a6ab0dab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:00.806563 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7ba458-21d4-4805-8c1f-c67a6ab0dab6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:00.819701 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7ba458-21d4-4805-8c1f-c67a6ab0dab6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:00.821588 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7ba458-21d4-4805-8c1f-c67a6ab0dab6 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:00.829728 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:00.842924 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-3e7ba458-21d4-4805-8c1f-c67a6ab0dab6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:01:00.842924 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:00.842924 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:00.842924 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:00.842924 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:00.842924 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:00.842924 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:00.842924 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:00.842924 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:00.842924 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:00.842924 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:00.842924 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:00.842924 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:01:00.842924 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:01:00.842924 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:01:00.843898 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:01:00.843898 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:00.843898 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:00.843898 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:00.843898 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:00.843898 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:00.843898 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:00.843898 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:00.843898 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:00.843898 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:00.843898 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:00.843898 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:00.843898 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:00.843898 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:00.843898 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:00.843898 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:00.843898 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:00.844750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:01:00.844750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:01:00.844750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:01:00.844750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:00.844750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 376/739] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:01:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:00.845546 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a254d4c9-3873-4f07-9d67-e1ea59279e14 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:00.845840 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a254d4c9-3873-4f07-9d67-e1ea59279e14 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:00.846116 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a254d4c9-3873-4f07-9d67-e1ea59279e14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:00.867141 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-a254d4c9-3873-4f07-9d67-e1ea59279e14 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:01:00.868376 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-a254d4c9-3873-4f07-9d67-e1ea59279e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:01:00.884660 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6905a38-df3e-4c2d-adc8-9331b4536278 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:00.923642 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc7d530-9f5a-49a2-97ff-7b077d89807e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:00.924305 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc7d530-9f5a-49a2-97ff-7b077d89807e None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:00.924590 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc7d530-9f5a-49a2-97ff-7b077d89807e None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:00.924868 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc7d530-9f5a-49a2-97ff-7b077d89807e None admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:00.925596 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc7d530-9f5a-49a2-97ff-7b077d89807e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:00.925902 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc7d530-9f5a-49a2-97ff-7b077d89807e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:00.927317 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc7d530-9f5a-49a2-97ff-7b077d89807e None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:00.944600 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 364/740] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:01:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:00.944600 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc7d530-9f5a-49a2-97ff-7b077d89807e None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:00.950154 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 377/741] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:01:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 19:01:00.951294 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a254d4c9-3873-4f07-9d67-e1ea59279e14 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:01.052898 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:01.056924 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a254d4c9-3873-4f07-9d67-e1ea59279e14 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:01.065953 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc7d530-9f5a-49a2-97ff-7b077d89807e None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:01.088557 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:01.097547 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89111ab-0a2e-447f-b207-8ad9749946b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:01.100890 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89111ab-0a2e-447f-b207-8ad9749946b9 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:01.100890 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89111ab-0a2e-447f-b207-8ad9749946b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:01.100890 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89111ab-0a2e-447f-b207-8ad9749946b9 None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:01.100890 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89111ab-0a2e-447f-b207-8ad9749946b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:01.100890 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89111ab-0a2e-447f-b207-8ad9749946b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:01.101447 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89111ab-0a2e-447f-b207-8ad9749946b9 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:01.103931 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ae4991-7c3c-4ec7-a6df-483c05bf5699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:01.105168 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae4991-7c3c-4ec7-a6df-483c05bf5699 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:01.105455 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae4991-7c3c-4ec7-a6df-483c05bf5699 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:01.105727 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae4991-7c3c-4ec7-a6df-483c05bf5699 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:01.114385 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 378/742] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:01:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:01.114385 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89111ab-0a2e-447f-b207-8ad9749946b9 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:01.122266 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ae4991-7c3c-4ec7-a6df-483c05bf5699 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:01.123769 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ae4991-7c3c-4ec7-a6df-483c05bf5699 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:01.142797 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-51ae4991-7c3c-4ec7-a6df-483c05bf5699 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:01:01.142797 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:01.142797 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:01.142797 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:01.142797 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:01.142797 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:01.142797 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:01.142797 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:01.142797 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:01.142797 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:01.142797 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:01.142797 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:01.142797 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:01:01.142797 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:01:01.142797 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:01:01.143728 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:01:01.143728 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:01.143728 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:01.143728 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:01.143728 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:01.143728 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:01.143728 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:01.143728 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:01.143728 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:01.143728 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:01.143728 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:01.143728 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:01.143728 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:01.143728 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:01.143728 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:01.143728 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:01.143728 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:01.144582 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:01:01.144582 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:01:01.144582 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:01:01.144582 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:01:01.144582 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 365/743] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:01:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:01.144869 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ae4991-7c3c-4ec7-a6df-483c05bf5699 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:01.230843 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89111ab-0a2e-447f-b207-8ad9749946b9 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:01.252656 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ae4991-7c3c-4ec7-a6df-483c05bf5699 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:01.268666 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b369f5d3-0d9c-469a-8d6d-886412912ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:01.269484 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b369f5d3-0d9c-469a-8d6d-886412912ea7 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:01.269765 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b369f5d3-0d9c-469a-8d6d-886412912ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:01.270039 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b369f5d3-0d9c-469a-8d6d-886412912ea7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:01.278598 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b369f5d3-0d9c-469a-8d6d-886412912ea7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:01.280105 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b369f5d3-0d9c-469a-8d6d-886412912ea7 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:01.288904 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35dfe49-e7fb-4d84-bf90-f9bb590e7823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:01.289608 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35dfe49-e7fb-4d84-bf90-f9bb590e7823 None admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:01.289896 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35dfe49-e7fb-4d84-bf90-f9bb590e7823 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:01.290215 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35dfe49-e7fb-4d84-bf90-f9bb590e7823 None admin] PATH_INFO: `/v3/services` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:01.290979 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35dfe49-e7fb-4d84-bf90-f9bb590e7823 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:01.292462 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35dfe49-e7fb-4d84-bf90-f9bb590e7823 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:01.294856 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-b369f5d3-0d9c-469a-8d6d-886412912ea7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 02 19:01:01.294856 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:01.294856 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:01.294856 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:01.294856 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:01.294856 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:01.294856 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:01.294856 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:01.294856 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:01.294856 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:01.294856 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:01.294856 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:01.294856 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 02 19:01:01.294856 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 02 19:01:01.294856 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 02 19:01:01.294856 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 02 19:01:01.295750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:01.295750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:01.295750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 02 19:01:01.295750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:01:01.295750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 02 19:01:01.295750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:01:01.295750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:01.295750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:01.295750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:01.295750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:01.295750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:01.295750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:01.295750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:01.295750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:01.295750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:01.295750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:01.295750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:01.295750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:01.296648 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:01.296648 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:01.296648 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 02 19:01:01.296648 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 02 19:01:01.296648 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 02 19:01:01.296648 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 02 19:01:01.296648 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 02 19:01:01.296648 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 02 19:01:01.296648 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 02 19:01:01.296648 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:01.296648 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 379/744] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Jun 2 19:01:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:01.297315 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b369f5d3-0d9c-469a-8d6d-886412912ea7 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:01.346216 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:01.392999 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:01.394172 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b369f5d3-0d9c-469a-8d6d-886412912ea7 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:01.398022 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 366/745] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Jun 2 19:01:01 2022] POST /identity/v3/services => generated 251 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:01:01.399406 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35dfe49-e7fb-4d84-bf90-f9bb590e7823 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:01.447013 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9002b0b5-b087-4f17-9ac2-28d1055ee8d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:01.448014 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9002b0b5-b087-4f17-9ac2-28d1055ee8d1 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:01.448305 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9002b0b5-b087-4f17-9ac2-28d1055ee8d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:01.448580 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9002b0b5-b087-4f17-9ac2-28d1055ee8d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:01.456168 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9002b0b5-b087-4f17-9ac2-28d1055ee8d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:01.457686 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9002b0b5-b087-4f17-9ac2-28d1055ee8d1 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:01.474462 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-9002b0b5-b087-4f17-9ac2-28d1055ee8d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:01:01.474462 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:01.474462 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:01.474462 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:01.474462 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:01.474462 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:01.474462 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:01.474462 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:01.474462 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:01.474462 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:01.474462 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:01.474462 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:01.474462 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 02 19:01:01.474462 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 02 19:01:01.474462 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 02 19:01:01.475810 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 02 19:01:01.475810 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:01.475810 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:01.475810 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:01.475810 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:01.475810 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:01.475810 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:01.475810 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:01.475810 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:01.475810 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:01.475810 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:01.475810 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:01.475810 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:01.475810 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:01.475810 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:01.475810 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:01.475810 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:01.477263 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 02 19:01:01.477263 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 02 19:01:01.477263 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 02 19:01:01.477263 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 02 19:01:01.477263 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 02 19:01:01.477263 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 02 19:01:01.477263 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:01:01.477263 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:01.477263 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 380/746] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 2 19:01:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:01.477263 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9002b0b5-b087-4f17-9ac2-28d1055ee8d1 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:01.516808 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35dfe49-e7fb-4d84-bf90-f9bb590e7823 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:01.554634 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497774f0-7478-46d5-bea4-032120b906d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:01.556063 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497774f0-7478-46d5-bea4-032120b906d8 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:01.556347 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497774f0-7478-46d5-bea4-032120b906d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:01.556626 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497774f0-7478-46d5-bea4-032120b906d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:01.557375 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497774f0-7478-46d5-bea4-032120b906d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:01.557682 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497774f0-7478-46d5-bea4-032120b906d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:01.559119 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497774f0-7478-46d5-bea4-032120b906d8 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:01.573207 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 367/747] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:01:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:01.573207 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee36580-2ad6-4f49-bcfd-467a48560862 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:01.573207 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee36580-2ad6-4f49-bcfd-467a48560862 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:01.573207 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee36580-2ad6-4f49-bcfd-467a48560862 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:01.573207 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 368/748] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:01:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:01:01.573207 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee36580-2ad6-4f49-bcfd-467a48560862 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:01.594091 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9002b0b5-b087-4f17-9ac2-28d1055ee8d1 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:01.597407 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:01.643140 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b03baf-c786-463d-a318-4492e53308fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:01.643924 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b03baf-c786-463d-a318-4492e53308fc None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:01.644211 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b03baf-c786-463d-a318-4492e53308fc None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:01.644380 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b03baf-c786-463d-a318-4492e53308fc None admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:01.645512 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b03baf-c786-463d-a318-4492e53308fc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:01.645512 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b03baf-c786-463d-a318-4492e53308fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:01.647070 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b03baf-c786-463d-a318-4492e53308fc None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:01.649032 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:01.665354 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 381/749] 10.222.0.33 () {56 vars in 1149 bytes} [Thu Jun 2 19:01:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:01.669961 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88e4f46-6dcc-46da-a47e-c6e254baa0b1 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:01.669961 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88e4f46-6dcc-46da-a47e-c6e254baa0b1 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:01.669961 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88e4f46-6dcc-46da-a47e-c6e254baa0b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:01.684139 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-b88e4f46-6dcc-46da-a47e-c6e254baa0b1 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:01:01.685333 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-b88e4f46-6dcc-46da-a47e-c6e254baa0b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:01:01.700071 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee36580-2ad6-4f49-bcfd-467a48560862 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:01.736675 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac576fa-8fe2-43eb-8d47-56f42fb788aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:01.737361 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac576fa-8fe2-43eb-8d47-56f42fb788aa None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:01.737643 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac576fa-8fe2-43eb-8d47-56f42fb788aa None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:01.737933 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac576fa-8fe2-43eb-8d47-56f42fb788aa None admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:01.738708 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac576fa-8fe2-43eb-8d47-56f42fb788aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:01.739048 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac576fa-8fe2-43eb-8d47-56f42fb788aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:01.740541 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac576fa-8fe2-43eb-8d47-56f42fb788aa None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:01.759284 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 369/750] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 2 19:01:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:01.759284 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac576fa-8fe2-43eb-8d47-56f42fb788aa None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:01.761303 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 382/751] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:01:01 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 02 19:01:01.762934 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88e4f46-6dcc-46da-a47e-c6e254baa0b1 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:01.872443 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:01.881147 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88e4f46-6dcc-46da-a47e-c6e254baa0b1 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:01.888747 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac576fa-8fe2-43eb-8d47-56f42fb788aa None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:01.911532 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:01.919724 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48afc125-44f6-4297-8418-99e3ae627d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:01.920378 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48afc125-44f6-4297-8418-99e3ae627d38 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:01.921969 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48afc125-44f6-4297-8418-99e3ae627d38 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:01.921969 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48afc125-44f6-4297-8418-99e3ae627d38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:01.927082 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a085608-601a-436d-978f-54ee770a48da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:01.927884 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a085608-601a-436d-978f-54ee770a48da None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:01.928167 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a085608-601a-436d-978f-54ee770a48da None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:01.928444 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a085608-601a-436d-978f-54ee770a48da None admin] PATH_INFO: `/v3/users/nova` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:01.937703 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a085608-601a-436d-978f-54ee770a48da None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:01.938869 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a085608-601a-436d-978f-54ee770a48da None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:01.955089 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-4a085608-601a-436d-978f-54ee770a48da None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 02 19:01:01.955089 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:01.955089 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:01.955089 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:01.955089 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:01.955089 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:01.955089 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:01.955089 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:01.955089 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:01.955089 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:01.955089 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:01.955089 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:01.955089 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 02 19:01:01.955089 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 02 19:01:01.955089 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 02 19:01:01.955089 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 02 19:01:01.956080 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:01.956080 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:01.956080 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 02 19:01:01.956080 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:01:01.956080 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 02 19:01:01.956080 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:01:01.956080 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:01.956080 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:01.956080 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:01.956080 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:01.956080 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:01.956080 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:01.956080 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:01.956080 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:01.956080 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:01.956080 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:01.956080 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:01.956080 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:01.956999 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:01.956999 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:01.956999 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 02 19:01:01.956999 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 02 19:01:01.956999 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 02 19:01:01.956999 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 02 19:01:01.956999 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 02 19:01:01.956999 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 02 19:01:01.956999 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 02 19:01:01.956999 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:01:01.956999 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 370/752] 10.222.0.33 () {56 vars in 1093 bytes} [Thu Jun 2 19:01:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:01.957585 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a085608-601a-436d-978f-54ee770a48da None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:01.997617 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48afc125-44f6-4297-8418-99e3ae627d38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:01.999127 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48afc125-44f6-4297-8418-99e3ae627d38 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:02.067319 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a085608-601a-436d-978f-54ee770a48da None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:02.095508 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:02.104215 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2d57b4-0177-4a11-8fed-3876508d7665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:02.113349 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2d57b4-0177-4a11-8fed-3876508d7665 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:02.113349 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2d57b4-0177-4a11-8fed-3876508d7665 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:02.113349 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2d57b4-0177-4a11-8fed-3876508d7665 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:02.113349 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2d57b4-0177-4a11-8fed-3876508d7665 None admin] RBAC: Adding query filter params (role.id=f13fdd0847924014abf1e95594195e78, scope.project.id=8c04020eaebd4647804cf8653f32d9e5, user.id=2a59e1d67ff94186a3ebcfcb49938a87) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:02.113349 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2d57b4-0177-4a11-8fed-3876508d7665 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:02.113738 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 383/753] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 2 19:01:01 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 02 19:01:02.113738 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48afc125-44f6-4297-8418-99e3ae627d38 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:02.113890 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2d57b4-0177-4a11-8fed-3876508d7665 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:02.114789 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 371/754] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 2 19:01:01 2022] GET /identity/v3/role_assignments?user.id=2a59e1d67ff94186a3ebcfcb49938a87&scope.project.id=8c04020eaebd4647804cf8653f32d9e5&role.id=f13fdd0847924014abf1e95594195e78 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:02.115419 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2d57b4-0177-4a11-8fed-3876508d7665 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:02.192565 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:02.233151 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48afc125-44f6-4297-8418-99e3ae627d38 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:02.275746 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2d57b4-0177-4a11-8fed-3876508d7665 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:02.281533 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79643494-24c9-4cf4-b863-fbe72579e796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:02.282339 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79643494-24c9-4cf4-b863-fbe72579e796 None admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:02.282699 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79643494-24c9-4cf4-b863-fbe72579e796 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:02.283073 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79643494-24c9-4cf4-b863-fbe72579e796 None admin] PATH_INFO: `/v3/services` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:02.283890 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79643494-24c9-4cf4-b863-fbe72579e796 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:02.285387 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79643494-24c9-4cf4-b863-fbe72579e796 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:02.318660 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d4939e-3bc7-4a12-a5d0-1e3b4304a09f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:02.319692 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d4939e-3bc7-4a12-a5d0-1e3b4304a09f None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:02.319976 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d4939e-3bc7-4a12-a5d0-1e3b4304a09f None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:02.320258 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d4939e-3bc7-4a12-a5d0-1e3b4304a09f None admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:02.321240 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d4939e-3bc7-4a12-a5d0-1e3b4304a09f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:02.321605 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d4939e-3bc7-4a12-a5d0-1e3b4304a09f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:02.323090 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d4939e-3bc7-4a12-a5d0-1e3b4304a09f None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:02.344565 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 372/755] 10.222.0.33 () {56 vars in 1133 bytes} [Thu Jun 2 19:01:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:02.344565 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d4939e-3bc7-4a12-a5d0-1e3b4304a09f None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:02.374119 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 384/756] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Jun 2 19:01:02 2022] POST /identity/v3/services => generated 240 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:01:02.374982 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79643494-24c9-4cf4-b863-fbe72579e796 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:02.438123 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:02.489150 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d4939e-3bc7-4a12-a5d0-1e3b4304a09f None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:02.491131 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:02.556358 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79643494-24c9-4cf4-b863-fbe72579e796 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:02.565456 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c6768e-5f12-4062-adb1-46b5cd444197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:02.566286 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c6768e-5f12-4062-adb1-46b5cd444197 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:02.566570 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c6768e-5f12-4062-adb1-46b5cd444197 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:02.566874 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c6768e-5f12-4062-adb1-46b5cd444197 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:02.580584 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c6768e-5f12-4062-adb1-46b5cd444197 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:02.582087 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c6768e-5f12-4062-adb1-46b5cd444197 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:02.608974 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-d3c6768e-5f12-4062-adb1-46b5cd444197 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:01:02.608974 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:02.608974 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:02.608974 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:02.608974 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:02.608974 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:02.608974 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:02.608974 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:02.608974 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:02.608974 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:02.608974 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:02.608974 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:02.608974 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:01:02.608974 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:01:02.608974 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:01:02.609997 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:01:02.609997 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:02.609997 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:02.609997 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:02.609997 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:02.609997 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:02.609997 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:02.609997 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:02.609997 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:02.609997 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:02.609997 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:02.609997 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:02.609997 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:02.609997 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:02.609997 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:02.609997 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:02.609997 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:02.610921 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:01:02.610921 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:01:02.610921 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:01:02.610921 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:01:02.617437 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 373/757] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:01:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:02.620528 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b466dd-19f8-4d47-8db0-ed2713ab4995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:02.621664 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b466dd-19f8-4d47-8db0-ed2713ab4995 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:02.621782 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c6768e-5f12-4062-adb1-46b5cd444197 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:02.622282 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b466dd-19f8-4d47-8db0-ed2713ab4995 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:02.622917 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b466dd-19f8-4d47-8db0-ed2713ab4995 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:02.643793 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b466dd-19f8-4d47-8db0-ed2713ab4995 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:02.646093 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b466dd-19f8-4d47-8db0-ed2713ab4995 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:02.669852 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-f2b466dd-19f8-4d47-8db0-ed2713ab4995 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:01:02.669852 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:02.669852 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:02.669852 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:02.669852 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:02.669852 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:02.669852 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:02.669852 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:02.669852 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:02.669852 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:02.669852 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:02.669852 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:02.669852 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:01:02.669852 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:01:02.669852 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:01:02.671010 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:01:02.671010 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:02.671010 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:02.671010 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:02.671010 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:02.671010 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:02.671010 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:02.671010 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:02.671010 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:02.671010 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:02.671010 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:02.671010 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:02.671010 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:02.671010 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:02.671010 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:02.671010 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:02.671010 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:02.672058 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:01:02.672058 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:01:02.672058 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:01:02.672058 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:02.674587 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 385/758] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:01:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:02.681401 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b466dd-19f8-4d47-8db0-ed2713ab4995 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:02.766358 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c6768e-5f12-4062-adb1-46b5cd444197 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:02.851195 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04201183-6084-430e-bf0a-982041a872ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:02.852409 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04201183-6084-430e-bf0a-982041a872ac None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:02.852693 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04201183-6084-430e-bf0a-982041a872ac None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:02.852969 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04201183-6084-430e-bf0a-982041a872ac None admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:02.853708 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04201183-6084-430e-bf0a-982041a872ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:02.854021 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04201183-6084-430e-bf0a-982041a872ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:02.855426 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04201183-6084-430e-bf0a-982041a872ac None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:02.857299 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:02.860869 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b466dd-19f8-4d47-8db0-ed2713ab4995 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:02.883982 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 374/759] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:01:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:02.889638 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04201183-6084-430e-bf0a-982041a872ac None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:02.911106 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:02.925121 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6374dd92-598a-4f5a-b574-ac007596dc82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:02.925801 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6374dd92-598a-4f5a-b574-ac007596dc82 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:02.926087 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6374dd92-598a-4f5a-b574-ac007596dc82 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:02.926367 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6374dd92-598a-4f5a-b574-ac007596dc82 None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:02.927448 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6374dd92-598a-4f5a-b574-ac007596dc82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:02.927768 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6374dd92-598a-4f5a-b574-ac007596dc82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:02.929148 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6374dd92-598a-4f5a-b574-ac007596dc82 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:02.953983 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 386/760] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:01:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:02.959879 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6374dd92-598a-4f5a-b574-ac007596dc82 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:03.050792 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04201183-6084-430e-bf0a-982041a872ac None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:03.092927 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d1c6f1-93eb-4ffb-be24-5c32c77b6456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:03.093734 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d1c6f1-93eb-4ffb-be24-5c32c77b6456 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:03.093996 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d1c6f1-93eb-4ffb-be24-5c32c77b6456 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:03.094267 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d1c6f1-93eb-4ffb-be24-5c32c77b6456 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:03.105892 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d1c6f1-93eb-4ffb-be24-5c32c77b6456 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:03.107399 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d1c6f1-93eb-4ffb-be24-5c32c77b6456 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:03.139718 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-91d1c6f1-93eb-4ffb-be24-5c32c77b6456 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:01:03.139718 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:03.139718 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:03.139718 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:03.139718 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:03.139718 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:03.139718 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:03.139718 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:03.139718 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:03.139718 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:03.139718 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:03.139718 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:03.139718 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 02 19:01:03.139718 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 02 19:01:03.139718 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 02 19:01:03.140616 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 02 19:01:03.140616 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:03.140616 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:03.140616 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:03.140616 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:03.140616 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:03.140616 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:03.140616 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:03.140616 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:03.140616 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:03.140616 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:03.140616 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:03.140616 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:03.140616 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:03.140616 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:03.140616 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:03.140616 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:03.141491 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 02 19:01:03.141491 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 02 19:01:03.141491 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 02 19:01:03.141491 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 02 19:01:03.141491 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 02 19:01:03.141491 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 02 19:01:03.141491 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:01:03.141491 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:01:03.141491 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 375/761] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 2 19:01:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:03.146409 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6374dd92-598a-4f5a-b574-ac007596dc82 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:03.147195 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d1c6f1-93eb-4ffb-be24-5c32c77b6456 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:03.210593 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74eb36e-ffac-44fb-ae9e-516cca0c308a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:03.211491 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74eb36e-ffac-44fb-ae9e-516cca0c308a None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:03.211882 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74eb36e-ffac-44fb-ae9e-516cca0c308a None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:03.212166 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74eb36e-ffac-44fb-ae9e-516cca0c308a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:03.219841 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:03.225293 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74eb36e-ffac-44fb-ae9e-516cca0c308a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:03.226821 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74eb36e-ffac-44fb-ae9e-516cca0c308a None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:03.241175 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-a74eb36e-ffac-44fb-ae9e-516cca0c308a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:01:03.241175 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:03.241175 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:03.241175 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:03.241175 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:03.241175 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:03.241175 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:03.241175 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:03.241175 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:03.241175 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:03.241175 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:03.241175 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:03.241175 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 02 19:01:03.241175 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 02 19:01:03.241175 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 02 19:01:03.243327 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 02 19:01:03.243327 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:03.243327 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:03.243327 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:03.243327 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:03.243327 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:03.243327 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:03.243327 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:03.243327 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:03.243327 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:03.243327 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:03.243327 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:03.243327 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:03.243327 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:03.243327 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:03.243327 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:03.243327 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:03.244345 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 02 19:01:03.244345 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 02 19:01:03.244345 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 02 19:01:03.244345 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 02 19:01:03.244345 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 02 19:01:03.244345 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 02 19:01:03.244345 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:01:03.244345 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:03.244779 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 387/762] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 2 19:01:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:03.256303 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74eb36e-ffac-44fb-ae9e-516cca0c308a None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:03.269111 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d1c6f1-93eb-4ffb-be24-5c32c77b6456 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:03.303114 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:03.308061 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf8207e-2a85-4068-9860-d08bffefb07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:03.308736 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf8207e-2a85-4068-9860-d08bffefb07e None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:03.309022 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf8207e-2a85-4068-9860-d08bffefb07e None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:03.309300 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf8207e-2a85-4068-9860-d08bffefb07e None admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:03.310093 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf8207e-2a85-4068-9860-d08bffefb07e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:03.310447 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf8207e-2a85-4068-9860-d08bffefb07e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:03.311896 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf8207e-2a85-4068-9860-d08bffefb07e None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:03.331754 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 376/763] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 2 19:01:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:03.338384 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf8207e-2a85-4068-9860-d08bffefb07e None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:03.420446 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74eb36e-ffac-44fb-ae9e-516cca0c308a None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:03.474873 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf8207e-2a85-4068-9860-d08bffefb07e None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:03.511494 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de301b9f-3f98-4c8f-a990-ba550b8bc69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:03.511711 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff11dee-a9b6-44d3-b7b8-86af5f1bd626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:03.512299 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de301b9f-3f98-4c8f-a990-ba550b8bc69c None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:03.512409 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff11dee-a9b6-44d3-b7b8-86af5f1bd626 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:03.512627 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff11dee-a9b6-44d3-b7b8-86af5f1bd626 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:03.512750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de301b9f-3f98-4c8f-a990-ba550b8bc69c None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:03.512906 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff11dee-a9b6-44d3-b7b8-86af5f1bd626 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:03.513001 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de301b9f-3f98-4c8f-a990-ba550b8bc69c None admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:03.513831 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de301b9f-3f98-4c8f-a990-ba550b8bc69c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:03.514147 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de301b9f-3f98-4c8f-a990-ba550b8bc69c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:03.514466 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:03.515648 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de301b9f-3f98-4c8f-a990-ba550b8bc69c None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:03.519190 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff11dee-a9b6-44d3-b7b8-86af5f1bd626 None admin] RBAC: Adding query filter params (role.id=0e56fba112f94ebba363ffab7f4aa9fe, scope.project.id=8c04020eaebd4647804cf8653f32d9e5, user.id=04f64813f09348eb91cb7bb2869f7612) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:03.519634 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff11dee-a9b6-44d3-b7b8-86af5f1bd626 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:03.521161 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff11dee-a9b6-44d3-b7b8-86af5f1bd626 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:03.529710 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 388/764] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 2 19:01:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:01:03.531722 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 377/765] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 2 19:01:03 2022] GET /identity/v3/role_assignments?user.id=04f64813f09348eb91cb7bb2869f7612&scope.project.id=8c04020eaebd4647804cf8653f32d9e5&role.id=0e56fba112f94ebba363ffab7f4aa9fe => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:01:03.547199 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de301b9f-3f98-4c8f-a990-ba550b8bc69c None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:03.669624 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:03.701959 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de301b9f-3f98-4c8f-a990-ba550b8bc69c None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:03.755211 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a41acb6-0eb8-4055-b757-d6e3ae88f809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:03.756201 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a41acb6-0eb8-4055-b757-d6e3ae88f809 None admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:03.756427 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a41acb6-0eb8-4055-b757-d6e3ae88f809 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:03.756681 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a41acb6-0eb8-4055-b757-d6e3ae88f809 None admin] PATH_INFO: `/v3/projects/8c04020eaebd4647804cf8653f32d9e5/users/70664d5cbdee47ee89c120761939cc75/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:03.760145 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a41acb6-0eb8-4055-b757-d6e3ae88f809 None admin] RBAC: Authorizing `identity:create_grant(project_id=8c04020eaebd4647804cf8653f32d9e5, user_id=70664d5cbdee47ee89c120761939cc75, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:03.761622 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a41acb6-0eb8-4055-b757-d6e3ae88f809 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:03.832911 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 389/766] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 2 19:01:03 2022] PUT /identity/v3/projects/8c04020eaebd4647804cf8653f32d9e5/users/70664d5cbdee47ee89c120761939cc75/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:01:03.993087 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae5269f-0c3a-4584-8a68-a9bc917648d6 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:03.993442 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae5269f-0c3a-4584-8a68-a9bc917648d6 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:03.993757 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae5269f-0c3a-4584-8a68-a9bc917648d6 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:03.995392 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 378/767] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:01:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:01:04.000854 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6d0f5c-2c74-4489-9c88-04a0893ca39a None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:04.001153 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6d0f5c-2c74-4489-9c88-04a0893ca39a None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:04.001426 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6d0f5c-2c74-4489-9c88-04a0893ca39a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:04.026321 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-fc6d0f5c-2c74-4489-9c88-04a0893ca39a None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:01:04.027684 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-fc6d0f5c-2c74-4489-9c88-04a0893ca39a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:01:04.137393 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 390/768] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:01:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 02 19:01:04.162868 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae5269f-0c3a-4584-8a68-a9bc917648d6 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:04.309367 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae5269f-0c3a-4584-8a68-a9bc917648d6 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:04.361204 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2907b64-1dde-495b-83f6-1d3adfc3704a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:04.362528 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2907b64-1dde-495b-83f6-1d3adfc3704a None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:04.363032 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2907b64-1dde-495b-83f6-1d3adfc3704a None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:04.363521 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2907b64-1dde-495b-83f6-1d3adfc3704a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:04.364390 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2907b64-1dde-495b-83f6-1d3adfc3704a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:04.365938 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2907b64-1dde-495b-83f6-1d3adfc3704a None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:04.382219 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-a2907b64-1dde-495b-83f6-1d3adfc3704a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 02 19:01:04.382219 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:04.382219 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 02 19:01:04.382219 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 02 19:01:04.382219 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 02 19:01:04.382219 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 02 19:01:04.382219 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 02 19:01:04.382219 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 02 19:01:04.382219 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 02 19:01:04.382219 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:01:04.382219 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 02 19:01:04.382219 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:01:04.382219 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:04.382219 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:04.382219 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:04.382219 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:04.383775 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:04.383775 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:04.383775 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:04.383775 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:04.383775 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:04.383775 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:04.383775 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:04.383775 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 02 19:01:04.383775 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 02 19:01:04.383775 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 02 19:01:04.383775 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 02 19:01:04.383775 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:04.383775 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:04.383775 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:04.383775 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:04.383775 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:04.384610 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:04.384610 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:04.384610 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:04.384610 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:04.384610 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:04.384610 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:04.384610 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:04.384610 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:04.384610 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:04.384610 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:04.384610 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:04.384610 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 02 19:01:04.384610 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 02 19:01:04.384610 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 02 19:01:04.384610 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:01:04.387614 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 379/769] 10.222.0.33 () {56 vars in 1129 bytes} [Thu Jun 2 19:01:04 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:04.393646 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6d0f5c-2c74-4489-9c88-04a0893ca39a None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:04.484796 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:04.527824 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6d0f5c-2c74-4489-9c88-04a0893ca39a None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:04.574653 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18fd6ca-80ad-4ebf-b6ad-66b729c4b2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:04.575276 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18fd6ca-80ad-4ebf-b6ad-66b729c4b2cc None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:04.575559 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18fd6ca-80ad-4ebf-b6ad-66b729c4b2cc None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:04.575836 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18fd6ca-80ad-4ebf-b6ad-66b729c4b2cc None admin] PATH_INFO: `/v3/services` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:04.576560 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18fd6ca-80ad-4ebf-b6ad-66b729c4b2cc None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:04.576901 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18fd6ca-80ad-4ebf-b6ad-66b729c4b2cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:04.578227 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18fd6ca-80ad-4ebf-b6ad-66b729c4b2cc None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:04.587736 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 391/770] 10.222.0.33 () {56 vars in 1124 bytes} [Thu Jun 2 19:01:04 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:04.593604 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2907b64-1dde-495b-83f6-1d3adfc3704a None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:04.703540 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:04.748072 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f193de-4163-4575-b664-8e3f5a4fa3e1 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:04.748591 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f193de-4163-4575-b664-8e3f5a4fa3e1 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:04.748668 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f193de-4163-4575-b664-8e3f5a4fa3e1 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:04.750168 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 392/771] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:01:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:01:04.764945 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2907b64-1dde-495b-83f6-1d3adfc3704a None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:04.793148 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cacafb0-0410-4d34-ae42-9831c41052ed None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:04.793463 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cacafb0-0410-4d34-ae42-9831c41052ed None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:04.793717 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cacafb0-0410-4d34-ae42-9831c41052ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:04.803989 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be318f97-d395-40b1-b47c-8f27615a9678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:04.804666 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be318f97-d395-40b1-b47c-8f27615a9678 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:04.805010 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be318f97-d395-40b1-b47c-8f27615a9678 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:04.805296 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be318f97-d395-40b1-b47c-8f27615a9678 None admin] PATH_INFO: `/v3/services` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:04.806024 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be318f97-d395-40b1-b47c-8f27615a9678 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:04.806332 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be318f97-d395-40b1-b47c-8f27615a9678 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:04.807742 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be318f97-d395-40b1-b47c-8f27615a9678 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:04.813452 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 380/772] 10.222.0.33 () {56 vars in 1124 bytes} [Thu Jun 2 19:01:04 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:04.823758 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be318f97-d395-40b1-b47c-8f27615a9678 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:04.835890 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-5cacafb0-0410-4d34-ae42-9831c41052ed None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:01:04.837587 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-5cacafb0-0410-4d34-ae42-9831c41052ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:01:04.967114 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be318f97-d395-40b1-b47c-8f27615a9678 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:04.986466 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 393/773] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:01:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 02 19:01:04.991434 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:05.006527 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05424160-a938-43c6-ab94-f1e6044cb032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:05.012724 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05424160-a938-43c6-ab94-f1e6044cb032 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:05.012724 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05424160-a938-43c6-ab94-f1e6044cb032 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:05.012724 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05424160-a938-43c6-ab94-f1e6044cb032 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:05.012724 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05424160-a938-43c6-ab94-f1e6044cb032 None admin] RBAC: Adding query filter params (interface=public, service_id=42a5286fc20541df897e26ef950b8776, region_id=RegionOne) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:05.013008 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05424160-a938-43c6-ab94-f1e6044cb032 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:05.014325 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05424160-a938-43c6-ab94-f1e6044cb032 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:05.031885 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 381/774] 10.222.0.33 () {56 vars in 1251 bytes} [Thu Jun 2 19:01:04 2022] GET /identity/v3/endpoints?service_id=42a5286fc20541df897e26ef950b8776&interface=public®ion_id=RegionOne => generated 194 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:05.035445 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cacafb0-0410-4d34-ae42-9831c41052ed None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:05.038427 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05424160-a938-43c6-ab94-f1e6044cb032 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:05.135141 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:05.201198 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cacafb0-0410-4d34-ae42-9831c41052ed None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:05.236042 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05424160-a938-43c6-ab94-f1e6044cb032 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:05.276762 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a7b5ca-65d5-4df6-831b-5fd8300868dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:05.277434 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a7b5ca-65d5-4df6-831b-5fd8300868dd None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:05.277721 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a7b5ca-65d5-4df6-831b-5fd8300868dd None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:05.278001 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a7b5ca-65d5-4df6-831b-5fd8300868dd None admin] PATH_INFO: `/v3/services` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:05.279169 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a7b5ca-65d5-4df6-831b-5fd8300868dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:05.280931 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a7b5ca-65d5-4df6-831b-5fd8300868dd None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:05.286870 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 382/775] 10.222.0.33 () {56 vars in 1087 bytes} [Thu Jun 2 19:01:05 2022] GET /identity/v3/services => generated 772 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:05.290245 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfcae90-eed6-480c-98f5-265c5492089d None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:05.290537 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfcae90-eed6-480c-98f5-265c5492089d None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:05.290864 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfcae90-eed6-480c-98f5-265c5492089d None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:05.291578 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58f1d10-1b34-4944-8a82-595e89cd37e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:05.292339 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 383/776] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:01:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 02 19:01:05.292827 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f1d10-1b34-4944-8a82-595e89cd37e8 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:05.293173 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f1d10-1b34-4944-8a82-595e89cd37e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:05.293514 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f1d10-1b34-4944-8a82-595e89cd37e8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:05.298870 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17a9cf-5915-4566-a7d8-3b8146936147 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:05.298870 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17a9cf-5915-4566-a7d8-3b8146936147 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:05.298870 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17a9cf-5915-4566-a7d8-3b8146936147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:05.316208 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-7b17a9cf-5915-4566-a7d8-3b8146936147 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:01:05.317790 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-7b17a9cf-5915-4566-a7d8-3b8146936147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:01:05.323756 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58f1d10-1b34-4944-8a82-595e89cd37e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:01:05.331624 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-d58f1d10-1b34-4944-8a82-595e89cd37e8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 02 19:01:05.331624 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:05.331624 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:05.331624 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:05.331624 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:05.331624 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:05.331624 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:05.331624 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:05.331624 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:05.331624 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:05.331624 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:05.331624 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:05.331624 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:01:05.331624 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:01:05.331624 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:01:05.331624 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:01:05.332582 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:01:05.332582 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:01:05.332582 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:01:05.332582 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:01:05.332582 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:05.332582 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:05.332582 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:05.332582 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:05.332582 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:05.332582 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:05.332582 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:05.332582 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:05.332582 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:05.332582 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:05.332582 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:05.332582 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:05.333442 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:05.333442 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:05.333442 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:05.333442 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:05.333442 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:01:05.333442 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:01:05.333442 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:01:05.333442 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:01:05.333442 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:01:05.333442 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:01:05.333442 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 02 19:01:05.333442 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:05.339199 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 394/777] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 2 19:01:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:05.345419 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58f1d10-1b34-4944-8a82-595e89cd37e8 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:05.397457 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:05.434197 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 384/778] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:01:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 02 19:01:05.458838 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b17a9cf-5915-4566-a7d8-3b8146936147 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:05.544839 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58f1d10-1b34-4944-8a82-595e89cd37e8 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:05.570996 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:05.593378 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae98de4-c65f-460e-b6f6-eae77f8d653f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:05.594065 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae98de4-c65f-460e-b6f6-eae77f8d653f None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:05.594349 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae98de4-c65f-460e-b6f6-eae77f8d653f None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:05.594622 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae98de4-c65f-460e-b6f6-eae77f8d653f None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:05.595409 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae98de4-c65f-460e-b6f6-eae77f8d653f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:05.595720 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae98de4-c65f-460e-b6f6-eae77f8d653f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:05.597146 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae98de4-c65f-460e-b6f6-eae77f8d653f None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:05.615572 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 395/779] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 2 19:01:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:05.623820 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae98de4-c65f-460e-b6f6-eae77f8d653f None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:05.648593 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b17a9cf-5915-4566-a7d8-3b8146936147 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:05.694295 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a43ed4-5913-40e2-afb2-90bd2a578705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:05.695235 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a43ed4-5913-40e2-afb2-90bd2a578705 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:05.695855 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a43ed4-5913-40e2-afb2-90bd2a578705 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:05.696198 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a43ed4-5913-40e2-afb2-90bd2a578705 None admin] PATH_INFO: `/v3/services/network` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:05.697503 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a43ed4-5913-40e2-afb2-90bd2a578705 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:05.699279 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a43ed4-5913-40e2-afb2-90bd2a578705 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:05.715456 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-44a43ed4-5913-40e2-afb2-90bd2a578705 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 02 19:01:05.715456 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:05.715456 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 02 19:01:05.715456 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 02 19:01:05.715456 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 02 19:01:05.715456 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 02 19:01:05.715456 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 02 19:01:05.715456 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 02 19:01:05.715456 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 02 19:01:05.715456 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:01:05.715456 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 02 19:01:05.715456 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:01:05.715456 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:05.715456 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:05.715456 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:05.715456 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:05.715456 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:05.716542 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:05.716542 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:05.716542 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:05.716542 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:05.716542 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:05.716542 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:05.716542 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 02 19:01:05.716542 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 02 19:01:05.716542 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 02 19:01:05.716542 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 02 19:01:05.716542 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:05.716542 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:05.716542 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:05.716542 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:05.716542 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:05.716542 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:05.717351 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:05.717351 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:05.717351 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:05.717351 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:05.717351 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:05.717351 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:05.717351 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:05.717351 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:05.717351 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:05.717351 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:05.717351 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 02 19:01:05.717351 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 02 19:01:05.717351 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 02 19:01:05.717351 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:01:05.717351 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 385/780] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 2 19:01:05 2022] GET /identity/v3/services/network => generated 88 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 02 19:01:05.723119 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a43ed4-5913-40e2-afb2-90bd2a578705 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:05.748516 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae98de4-c65f-460e-b6f6-eae77f8d653f None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:05.779136 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:05.832144 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05a9265-903b-463b-9cb5-bdc19d434834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:05.832962 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a9265-903b-463b-9cb5-bdc19d434834 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:05.833256 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a9265-903b-463b-9cb5-bdc19d434834 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:05.833532 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a9265-903b-463b-9cb5-bdc19d434834 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:05.839303 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a43ed4-5913-40e2-afb2-90bd2a578705 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:05.869513 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05a9265-903b-463b-9cb5-bdc19d434834 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:05.871148 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05a9265-903b-463b-9cb5-bdc19d434834 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:05.898213 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-f05a9265-903b-463b-9cb5-bdc19d434834 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:01:05.898213 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:05.898213 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:05.898213 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:05.898213 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:05.898213 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:05.898213 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:05.898213 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:05.898213 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:05.898213 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:05.898213 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:05.898213 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:05.898213 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:01:05.898213 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:01:05.898213 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:01:05.899323 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:01:05.899323 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:05.899323 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:05.899323 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:05.899323 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:05.899323 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:05.899323 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:05.899323 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:05.899323 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:05.899323 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:05.899323 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:05.899323 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:05.899323 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:05.899323 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:05.899323 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:05.899323 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:05.899323 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:05.900213 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:01:05.900213 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:01:05.900213 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:01:05.900213 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:05.901482 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87278046-72d8-4788-a4a5-05a958c1623d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:05.902112 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87278046-72d8-4788-a4a5-05a958c1623d None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:05.902399 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87278046-72d8-4788-a4a5-05a958c1623d None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:05.902675 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87278046-72d8-4788-a4a5-05a958c1623d None admin] PATH_INFO: `/v3/services` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:05.903570 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87278046-72d8-4788-a4a5-05a958c1623d None admin] RBAC: Adding query filter params (name=network) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:05.903861 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87278046-72d8-4788-a4a5-05a958c1623d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:05.905226 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87278046-72d8-4788-a4a5-05a958c1623d None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:05.908073 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 396/781] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:01:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:05.911684 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 386/782] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Jun 2 19:01:05 2022] GET /identity/v3/services?name=network => generated 124 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:05.912569 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05a9265-903b-463b-9cb5-bdc19d434834 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:05.917288 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87278046-72d8-4788-a4a5-05a958c1623d None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:05.995015 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:06.087877 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05a9265-903b-463b-9cb5-bdc19d434834 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:06.106699 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87278046-72d8-4788-a4a5-05a958c1623d None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:06.127213 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:06.141253 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f6534d-ae29-43b5-b63c-c6337bd7d80d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:06.141956 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f6534d-ae29-43b5-b63c-c6337bd7d80d None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:06.142243 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f6534d-ae29-43b5-b63c-c6337bd7d80d None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:06.142521 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f6534d-ae29-43b5-b63c-c6337bd7d80d None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:06.143285 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f6534d-ae29-43b5-b63c-c6337bd7d80d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:06.143645 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f6534d-ae29-43b5-b63c-c6337bd7d80d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:06.145035 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f6534d-ae29-43b5-b63c-c6337bd7d80d None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:06.152807 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2de7e6e-a511-46e7-9535-cdedd470e880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:06.153417 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2de7e6e-a511-46e7-9535-cdedd470e880 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:06.153704 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2de7e6e-a511-46e7-9535-cdedd470e880 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:06.153984 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2de7e6e-a511-46e7-9535-cdedd470e880 None admin] PATH_INFO: `/v3/services` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:06.154701 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2de7e6e-a511-46e7-9535-cdedd470e880 None admin] RBAC: Adding query filter params (type=network) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:06.155034 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2de7e6e-a511-46e7-9535-cdedd470e880 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:06.156408 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2de7e6e-a511-46e7-9535-cdedd470e880 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:06.159826 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 397/783] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:01:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:01:06.161804 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 387/784] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Jun 2 19:01:05 2022] GET /identity/v3/services?type=network => generated 350 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:06.165830 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f6534d-ae29-43b5-b63c-c6337bd7d80d None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:06.168030 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2de7e6e-a511-46e7-9535-cdedd470e880 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:06.273364 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2de7e6e-a511-46e7-9535-cdedd470e880 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:06.279058 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:06.283587 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f6534d-ae29-43b5-b63c-c6337bd7d80d None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:06.329445 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641e356c-bded-489d-ab7f-94e478a58ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:06.330257 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641e356c-bded-489d-ab7f-94e478a58ac7 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:06.330539 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641e356c-bded-489d-ab7f-94e478a58ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:06.330836 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641e356c-bded-489d-ab7f-94e478a58ac7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:06.334308 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b28d67e-9a08-4ae4-ab1a-8d7a945e56d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:06.334928 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b28d67e-9a08-4ae4-ab1a-8d7a945e56d2 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:06.335212 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b28d67e-9a08-4ae4-ab1a-8d7a945e56d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:06.335491 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b28d67e-9a08-4ae4-ab1a-8d7a945e56d2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:06.336315 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b28d67e-9a08-4ae4-ab1a-8d7a945e56d2 None admin] RBAC: Adding query filter params (interface=public, service_id=a76cd5cf08ff4f9eaaecdaf3d408022a, region_id=RegionOne) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:06.336632 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b28d67e-9a08-4ae4-ab1a-8d7a945e56d2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:06.338023 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b28d67e-9a08-4ae4-ab1a-8d7a945e56d2 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:06.346738 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641e356c-bded-489d-ab7f-94e478a58ac7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:06.349263 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641e356c-bded-489d-ab7f-94e478a58ac7 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:06.352653 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 388/785] 10.222.0.33 () {56 vars in 1251 bytes} [Thu Jun 2 19:01:06 2022] GET /identity/v3/endpoints?service_id=a76cd5cf08ff4f9eaaecdaf3d408022a&interface=public®ion_id=RegionOne => generated 194 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:01:06.357796 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b28d67e-9a08-4ae4-ab1a-8d7a945e56d2 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:06.379655 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-641e356c-bded-489d-ab7f-94e478a58ac7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 02 19:01:06.379655 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:06.379655 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:06.379655 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:06.379655 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:06.379655 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:06.379655 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:06.379655 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:06.379655 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:06.379655 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:06.379655 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:06.379655 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:06.379655 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 02 19:01:06.379655 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 02 19:01:06.379655 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 02 19:01:06.379655 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 02 19:01:06.381827 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:06.381827 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:06.381827 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 02 19:01:06.381827 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:01:06.381827 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 02 19:01:06.381827 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:01:06.381827 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:06.381827 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:06.381827 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:06.381827 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:06.381827 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:06.381827 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:06.381827 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:06.381827 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:06.381827 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:06.381827 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:06.381827 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:06.381827 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:06.384674 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:06.384674 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:06.384674 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 02 19:01:06.384674 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 02 19:01:06.384674 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 02 19:01:06.384674 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 02 19:01:06.384674 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 02 19:01:06.384674 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 02 19:01:06.384674 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 02 19:01:06.384674 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:06.390265 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 398/786] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 2 19:01:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 02 19:01:06.390265 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641e356c-bded-489d-ab7f-94e478a58ac7 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:06.435888 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:06.513030 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641e356c-bded-489d-ab7f-94e478a58ac7 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:06.551392 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a70756c-f30f-4eda-a50b-057a3c098e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:06.552078 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a70756c-f30f-4eda-a50b-057a3c098e50 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:06.552365 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a70756c-f30f-4eda-a50b-057a3c098e50 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:06.552551 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b28d67e-9a08-4ae4-ab1a-8d7a945e56d2 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:06.552723 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a70756c-f30f-4eda-a50b-057a3c098e50 None admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:06.553484 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a70756c-f30f-4eda-a50b-057a3c098e50 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:06.553926 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a70756c-f30f-4eda-a50b-057a3c098e50 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:06.555541 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a70756c-f30f-4eda-a50b-057a3c098e50 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:06.585603 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 399/787] 10.222.0.33 () {56 vars in 1135 bytes} [Thu Jun 2 19:01:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:06.591579 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a70756c-f30f-4eda-a50b-057a3c098e50 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:06.598501 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831eec8c-8cc1-4f6e-b953-22dc22564e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:06.599203 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831eec8c-8cc1-4f6e-b953-22dc22564e7d None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:06.599490 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831eec8c-8cc1-4f6e-b953-22dc22564e7d None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:06.599766 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831eec8c-8cc1-4f6e-b953-22dc22564e7d None admin] PATH_INFO: `/v3/services` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:06.600457 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831eec8c-8cc1-4f6e-b953-22dc22564e7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:06.601888 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831eec8c-8cc1-4f6e-b953-22dc22564e7d None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:06.607920 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 389/788] 10.222.0.33 () {56 vars in 1087 bytes} [Thu Jun 2 19:01:06 2022] GET /identity/v3/services => generated 772 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:01:06.626025 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4a2071-d4bb-41b1-a18f-c6fb7b77a5de None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:06.626361 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4a2071-d4bb-41b1-a18f-c6fb7b77a5de None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:06.626653 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4a2071-d4bb-41b1-a18f-c6fb7b77a5de None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:06.626989 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:06.628441 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 390/789] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:01:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 02 19:01:06.635448 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce13ff2-c969-4a02-8446-a85522ed2f99 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:06.635727 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce13ff2-c969-4a02-8446-a85522ed2f99 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:06.636004 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce13ff2-c969-4a02-8446-a85522ed2f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:06.683992 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-5ce13ff2-c969-4a02-8446-a85522ed2f99 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:01:06.685278 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-5ce13ff2-c969-4a02-8446-a85522ed2f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:01:06.737063 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a70756c-f30f-4eda-a50b-057a3c098e50 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:06.781796 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30daad25-0b67-4a3a-bab8-d5b53d492cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:06.783455 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30daad25-0b67-4a3a-bab8-d5b53d492cf1 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:06.783960 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30daad25-0b67-4a3a-bab8-d5b53d492cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:06.784480 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30daad25-0b67-4a3a-bab8-d5b53d492cf1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:06.797905 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30daad25-0b67-4a3a-bab8-d5b53d492cf1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:06.799833 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30daad25-0b67-4a3a-bab8-d5b53d492cf1 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:06.801765 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 391/790] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:01:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 02 19:01:06.811279 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d03e1-0291-4420-ac28-ee1647cd1e88 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:06.811279 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d03e1-0291-4420-ac28-ee1647cd1e88 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:06.811279 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d03e1-0291-4420-ac28-ee1647cd1e88 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:06.811279 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 392/791] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:01:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:01:06.812357 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e2dc5a-fab2-4b38-94ed-3df946f0a845 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:06.812744 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e2dc5a-fab2-4b38-94ed-3df946f0a845 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:06.813094 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e2dc5a-fab2-4b38-94ed-3df946f0a845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:06.833988 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-b9e2dc5a-fab2-4b38-94ed-3df946f0a845 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:01:06.835470 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-b9e2dc5a-fab2-4b38-94ed-3df946f0a845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:01:06.839769 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-30daad25-0b67-4a3a-bab8-d5b53d492cf1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:01:06.839769 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:06.839769 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:06.839769 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:06.839769 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:06.839769 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:06.839769 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:06.839769 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:06.839769 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:06.839769 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:06.839769 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:06.839769 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:06.839769 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:01:06.839769 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:01:06.839769 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:01:06.840735 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:01:06.840735 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:06.840735 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:06.840735 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:06.840735 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:06.840735 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:06.840735 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:06.840735 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:06.840735 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:06.840735 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:06.840735 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:06.840735 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:06.840735 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:06.840735 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:06.840735 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:06.840735 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:06.840735 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:06.841599 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:01:06.841599 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:01:06.841599 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:01:06.841599 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:06.851458 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 400/792] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:01:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 02 19:01:06.853324 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30daad25-0b67-4a3a-bab8-d5b53d492cf1 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:06.870049 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:06.934333 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 393/793] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:01:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 02 19:01:06.935280 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e2dc5a-fab2-4b38-94ed-3df946f0a845 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:06.955200 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:07.006797 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30daad25-0b67-4a3a-bab8-d5b53d492cf1 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:07.055768 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e2dc5a-fab2-4b38-94ed-3df946f0a845 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:07.059693 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010a1132-564a-4bd6-9ee9-8dc61996d9b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:07.060379 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a1132-564a-4bd6-9ee9-8dc61996d9b1 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:07.060660 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a1132-564a-4bd6-9ee9-8dc61996d9b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:07.060932 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a1132-564a-4bd6-9ee9-8dc61996d9b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:07.061670 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010a1132-564a-4bd6-9ee9-8dc61996d9b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:07.061975 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010a1132-564a-4bd6-9ee9-8dc61996d9b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:07.063427 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010a1132-564a-4bd6-9ee9-8dc61996d9b1 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:07.077105 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 401/794] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:01:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:07.078131 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010a1132-564a-4bd6-9ee9-8dc61996d9b1 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:07.107814 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9880c67c-74b4-4f42-987f-51da24593ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:07.109521 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9880c67c-74b4-4f42-987f-51da24593ee4 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:07.110332 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9880c67c-74b4-4f42-987f-51da24593ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:07.110854 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9880c67c-74b4-4f42-987f-51da24593ee4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:07.127099 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9880c67c-74b4-4f42-987f-51da24593ee4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:01:07.138194 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-9880c67c-74b4-4f42-987f-51da24593ee4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 02 19:01:07.138194 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:07.138194 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:07.138194 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:07.138194 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:07.138194 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:07.138194 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:07.138194 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:07.138194 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:07.138194 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:07.138194 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:07.138194 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:07.138194 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:01:07.138194 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:01:07.138194 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:01:07.138194 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:01:07.139302 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:01:07.139302 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:01:07.139302 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:01:07.139302 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:01:07.139302 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:07.139302 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:07.139302 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:07.139302 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:07.139302 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:07.139302 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:07.139302 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:07.139302 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:07.139302 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:07.139302 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:07.139302 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:07.139302 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:07.140117 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:07.140117 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:07.140117 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:07.140117 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:07.140117 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:01:07.140117 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:01:07.140117 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:01:07.140117 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:01:07.140117 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:01:07.140117 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:01:07.140117 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 02 19:01:07.140117 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:01:07.140742 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 394/795] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 2 19:01:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:07.141400 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9880c67c-74b4-4f42-987f-51da24593ee4 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:07.169026 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:07.205336 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010a1132-564a-4bd6-9ee9-8dc61996d9b1 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:07.236139 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:07.245686 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd92aa0e-fec7-4a11-b73f-14f16bacf2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:07.248132 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd92aa0e-fec7-4a11-b73f-14f16bacf2bb None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:07.248132 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd92aa0e-fec7-4a11-b73f-14f16bacf2bb None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:07.248132 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd92aa0e-fec7-4a11-b73f-14f16bacf2bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:07.253636 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9880c67c-74b4-4f42-987f-51da24593ee4 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:07.260512 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd92aa0e-fec7-4a11-b73f-14f16bacf2bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:01:07.269927 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-dd92aa0e-fec7-4a11-b73f-14f16bacf2bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 02 19:01:07.269927 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:07.269927 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:07.269927 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:07.269927 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:07.269927 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:07.269927 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:07.269927 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:07.269927 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:07.269927 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:07.269927 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:07.269927 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:07.269927 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:01:07.269927 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:01:07.269927 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:01:07.269927 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:01:07.270867 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:01:07.270867 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:01:07.270867 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:01:07.270867 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:01:07.270867 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:07.270867 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:07.270867 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:07.270867 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:07.270867 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:07.270867 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:07.270867 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:07.270867 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:07.270867 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:07.270867 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:07.270867 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:07.270867 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:07.271704 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:07.271704 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:07.271704 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:07.271704 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:07.271704 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:01:07.271704 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:01:07.271704 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:01:07.271704 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:01:07.271704 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:01:07.271704 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:01:07.271704 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 02 19:01:07.271704 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:07.274143 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 402/796] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 2 19:01:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:07.275209 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd92aa0e-fec7-4a11-b73f-14f16bacf2bb None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:07.295622 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4e33e3-5c9f-4375-87c4-ce02636b19df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:07.296424 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4e33e3-5c9f-4375-87c4-ce02636b19df None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:07.296707 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4e33e3-5c9f-4375-87c4-ce02636b19df None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:07.296984 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4e33e3-5c9f-4375-87c4-ce02636b19df None admin] PATH_INFO: `/v3/projects/service` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:07.306215 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4e33e3-5c9f-4375-87c4-ce02636b19df None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:07.307670 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4e33e3-5c9f-4375-87c4-ce02636b19df None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:07.322047 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-3b4e33e3-5c9f-4375-87c4-ce02636b19df None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:01:07.322047 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:07.322047 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:07.322047 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:07.322047 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:07.322047 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:07.322047 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:07.322047 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:07.322047 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:07.322047 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:07.322047 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:07.322047 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:07.322047 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 02 19:01:07.322047 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 02 19:01:07.322047 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 02 19:01:07.323079 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 02 19:01:07.323079 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:07.323079 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:07.323079 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:07.323079 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:07.323079 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:07.323079 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:07.323079 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:07.323079 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:07.323079 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:07.323079 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:07.323079 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:07.323079 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:07.323079 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:07.323079 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:07.323079 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:07.323079 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:07.324067 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 02 19:01:07.324067 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 02 19:01:07.324067 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 02 19:01:07.324067 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 02 19:01:07.324067 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 02 19:01:07.324067 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 02 19:01:07.324067 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:01:07.324067 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:01:07.324067 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 395/797] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 2 19:01:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:07.324559 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4e33e3-5c9f-4375-87c4-ce02636b19df None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:07.391896 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd92aa0e-fec7-4a11-b73f-14f16bacf2bb None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:07.409319 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:07.430408 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4e33e3-5c9f-4375-87c4-ce02636b19df None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:07.431346 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76edea23-3de8-4405-a541-503efbf5c5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:07.432272 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76edea23-3de8-4405-a541-503efbf5c5c3 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:07.432758 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76edea23-3de8-4405-a541-503efbf5c5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:07.433229 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76edea23-3de8-4405-a541-503efbf5c5c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:07.434197 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76edea23-3de8-4405-a541-503efbf5c5c3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:07.434726 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76edea23-3de8-4405-a541-503efbf5c5c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:07.436379 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76edea23-3de8-4405-a541-503efbf5c5c3 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:07.447850 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 403/798] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 2 19:01:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:07.449081 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76edea23-3de8-4405-a541-503efbf5c5c3 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:07.472644 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4968b0cc-25d1-4e64-b7ab-ad716b2b7af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:07.473320 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4968b0cc-25d1-4e64-b7ab-ad716b2b7af9 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:07.473600 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4968b0cc-25d1-4e64-b7ab-ad716b2b7af9 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:07.473874 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4968b0cc-25d1-4e64-b7ab-ad716b2b7af9 None admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:07.474943 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4968b0cc-25d1-4e64-b7ab-ad716b2b7af9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:07.474943 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4968b0cc-25d1-4e64-b7ab-ad716b2b7af9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:07.476329 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4968b0cc-25d1-4e64-b7ab-ad716b2b7af9 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:07.490334 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 396/799] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 2 19:01:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:07.491332 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4968b0cc-25d1-4e64-b7ab-ad716b2b7af9 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:07.507150 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:07.564272 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76edea23-3de8-4405-a541-503efbf5c5c3 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:07.597758 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4968b0cc-25d1-4e64-b7ab-ad716b2b7af9 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:07.603669 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97516b43-f1dc-44dc-9a2f-c923d1f966a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:07.604634 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97516b43-f1dc-44dc-9a2f-c923d1f966a2 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:07.605134 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97516b43-f1dc-44dc-9a2f-c923d1f966a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:07.605713 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97516b43-f1dc-44dc-9a2f-c923d1f966a2 None admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:07.606856 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97516b43-f1dc-44dc-9a2f-c923d1f966a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:07.607423 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97516b43-f1dc-44dc-9a2f-c923d1f966a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:07.609224 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97516b43-f1dc-44dc-9a2f-c923d1f966a2 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:07.623895 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 404/800] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 2 19:01:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:07.625178 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97516b43-f1dc-44dc-9a2f-c923d1f966a2 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:07.634595 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d843c5-0717-4014-8dc6-0330ce6f0f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:07.635581 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d843c5-0717-4014-8dc6-0330ce6f0f3e None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:07.636076 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d843c5-0717-4014-8dc6-0330ce6f0f3e None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:07.636532 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d843c5-0717-4014-8dc6-0330ce6f0f3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:07.648239 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d843c5-0717-4014-8dc6-0330ce6f0f3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:07.651609 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d843c5-0717-4014-8dc6-0330ce6f0f3e None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:07.653979 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:07.670776 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-30d843c5-0717-4014-8dc6-0330ce6f0f3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:01:07.670776 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:07.670776 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:07.670776 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:07.670776 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:07.670776 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:07.670776 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:07.670776 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:07.670776 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:07.670776 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:07.670776 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:07.670776 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:07.670776 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:01:07.670776 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:01:07.670776 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:01:07.671722 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:01:07.671722 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:07.671722 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:07.671722 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:07.671722 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:07.671722 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:07.671722 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:07.671722 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:07.671722 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:07.671722 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:07.671722 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:07.671722 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:07.671722 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:07.671722 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:07.671722 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:07.671722 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:07.671722 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:07.672600 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:01:07.672600 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:01:07.672600 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:01:07.672600 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:01:07.679180 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 397/801] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:01:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:07.680440 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d843c5-0717-4014-8dc6-0330ce6f0f3e None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:07.743597 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97516b43-f1dc-44dc-9a2f-c923d1f966a2 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:07.755787 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:07.782230 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d33289-de40-4c5d-9349-9b657c5f5818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:07.783107 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d33289-de40-4c5d-9349-9b657c5f5818 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:07.783679 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d33289-de40-4c5d-9349-9b657c5f5818 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:07.783795 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d33289-de40-4c5d-9349-9b657c5f5818 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:07.793255 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d843c5-0717-4014-8dc6-0330ce6f0f3e None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:07.796952 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d33289-de40-4c5d-9349-9b657c5f5818 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:07.798130 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d33289-de40-4c5d-9349-9b657c5f5818 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:07.817610 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-19d33289-de40-4c5d-9349-9b657c5f5818 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:01:07.817610 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:07.817610 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:07.817610 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:07.817610 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:07.817610 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:07.817610 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:07.817610 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:07.817610 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:07.817610 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:07.817610 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:07.817610 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:07.817610 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:01:07.817610 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:01:07.817610 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:01:07.818517 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:01:07.818517 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:07.818517 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:07.818517 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:07.818517 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:07.818517 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:07.818517 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:07.818517 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:07.818517 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:07.818517 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:07.818517 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:07.818517 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:07.818517 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:07.818517 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:07.818517 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:07.818517 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:07.818517 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:07.819496 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:01:07.819496 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:01:07.819496 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:01:07.819496 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:07.823473 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 405/802] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:01:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:07.824468 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d33289-de40-4c5d-9349-9b657c5f5818 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:07.833621 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5369a2f1-7538-463a-9fb6-ba6f88d1bde4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:07.834794 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5369a2f1-7538-463a-9fb6-ba6f88d1bde4 None admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:07.835224 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5369a2f1-7538-463a-9fb6-ba6f88d1bde4 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:07.835660 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5369a2f1-7538-463a-9fb6-ba6f88d1bde4 None admin] PATH_INFO: `/v3/projects/8c04020eaebd4647804cf8653f32d9e5/users/2a59e1d67ff94186a3ebcfcb49938a87/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:07.838320 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5369a2f1-7538-463a-9fb6-ba6f88d1bde4 None admin] RBAC: Authorizing `identity:create_grant(project_id=8c04020eaebd4647804cf8653f32d9e5, user_id=2a59e1d67ff94186a3ebcfcb49938a87, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:07.840015 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5369a2f1-7538-463a-9fb6-ba6f88d1bde4 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:07.910965 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 398/803] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 2 19:01:07 2022] PUT /identity/v3/projects/8c04020eaebd4647804cf8653f32d9e5/users/2a59e1d67ff94186a3ebcfcb49938a87/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:01:07.913005 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5369a2f1-7538-463a-9fb6-ba6f88d1bde4 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:07.938088 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d33289-de40-4c5d-9349-9b657c5f5818 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:07.983078 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe79daa-a688-477e-b435-16e58ff7f2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:07.983078 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe79daa-a688-477e-b435-16e58ff7f2b2 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:07.983078 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe79daa-a688-477e-b435-16e58ff7f2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:07.983078 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe79daa-a688-477e-b435-16e58ff7f2b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:07.983519 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe79daa-a688-477e-b435-16e58ff7f2b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:07.983519 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe79daa-a688-477e-b435-16e58ff7f2b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:07.983519 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe79daa-a688-477e-b435-16e58ff7f2b2 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:07.995496 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 406/804] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:01:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:08.007409 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe79daa-a688-477e-b435-16e58ff7f2b2 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:08.023257 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:08.045843 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:08.088384 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5369a2f1-7538-463a-9fb6-ba6f88d1bde4 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:08.131417 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04bc482-c349-46c8-aedf-ec425e6335b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:08.132175 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04bc482-c349-46c8-aedf-ec425e6335b2 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:08.132665 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04bc482-c349-46c8-aedf-ec425e6335b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:08.132976 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04bc482-c349-46c8-aedf-ec425e6335b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:08.134353 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04bc482-c349-46c8-aedf-ec425e6335b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:08.134822 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04bc482-c349-46c8-aedf-ec425e6335b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:08.136444 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04bc482-c349-46c8-aedf-ec425e6335b2 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:08.151349 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 399/805] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:01:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:08.157926 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea574dd-4bf6-40d3-b74b-218c2b265e1d None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:08.158358 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea574dd-4bf6-40d3-b74b-218c2b265e1d None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:08.158736 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea574dd-4bf6-40d3-b74b-218c2b265e1d None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:08.165283 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 400/806] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:01:08 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:01:08.165283 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea574dd-4bf6-40d3-b74b-218c2b265e1d None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:08.170214 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe79daa-a688-477e-b435-16e58ff7f2b2 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:08.220667 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ee9768-615c-44fc-912e-ab228443f13c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:08.221541 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ee9768-615c-44fc-912e-ab228443f13c None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:08.221795 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ee9768-615c-44fc-912e-ab228443f13c None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:08.222072 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ee9768-615c-44fc-912e-ab228443f13c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:08.230388 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ee9768-615c-44fc-912e-ab228443f13c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:08.231900 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ee9768-615c-44fc-912e-ab228443f13c None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:08.248266 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-b9ee9768-615c-44fc-912e-ab228443f13c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 02 19:01:08.248266 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:08.248266 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:08.248266 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:08.248266 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:08.248266 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:08.248266 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:08.248266 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:08.248266 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:08.248266 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:08.248266 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:08.248266 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:08.248266 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 02 19:01:08.248266 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 02 19:01:08.248266 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 02 19:01:08.248266 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 02 19:01:08.249404 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:08.249404 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:08.249404 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 02 19:01:08.249404 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:01:08.249404 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 02 19:01:08.249404 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:01:08.249404 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:08.249404 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:08.249404 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:08.249404 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:08.249404 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:08.249404 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:08.249404 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:08.249404 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:08.249404 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:08.249404 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:08.249404 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:08.249404 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:08.250391 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:08.250391 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:08.250391 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 02 19:01:08.250391 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 02 19:01:08.250391 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 02 19:01:08.250391 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 02 19:01:08.250391 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 02 19:01:08.250391 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 02 19:01:08.250391 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 02 19:01:08.250391 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:08.250391 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 407/807] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Jun 2 19:01:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:08.250997 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c0b192-d832-4a39-a1d1-ca6881ecab6d None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:08.250997 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c0b192-d832-4a39-a1d1-ca6881ecab6d None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:08.251275 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c0b192-d832-4a39-a1d1-ca6881ecab6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:08.268853 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-99c0b192-d832-4a39-a1d1-ca6881ecab6d None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:01:08.270041 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-99c0b192-d832-4a39-a1d1-ca6881ecab6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:01:08.292764 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea574dd-4bf6-40d3-b74b-218c2b265e1d None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:08.319130 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:08.337123 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28329f26-23e5-407f-8b89-04d81ceb6cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:08.338159 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28329f26-23e5-407f-8b89-04d81ceb6cff None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:08.338639 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28329f26-23e5-407f-8b89-04d81ceb6cff None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:08.339334 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28329f26-23e5-407f-8b89-04d81ceb6cff None admin] PATH_INFO: `/v3/users/nova` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:08.349838 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28329f26-23e5-407f-8b89-04d81ceb6cff None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:08.351574 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28329f26-23e5-407f-8b89-04d81ceb6cff None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:08.353597 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 408/808] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:01:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 02 19:01:08.354708 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c0b192-d832-4a39-a1d1-ca6881ecab6d None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:08.371249 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:08.372825 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-28329f26-23e5-407f-8b89-04d81ceb6cff None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 02 19:01:08.372825 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:08.372825 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:08.372825 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:08.372825 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:08.372825 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:08.372825 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:08.372825 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:08.372825 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:08.372825 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:08.372825 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:08.372825 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:08.372825 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 02 19:01:08.372825 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 02 19:01:08.372825 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 02 19:01:08.372825 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 02 19:01:08.373727 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:08.373727 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:08.373727 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 02 19:01:08.373727 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:01:08.373727 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 02 19:01:08.373727 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:01:08.373727 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:08.373727 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:08.373727 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:08.373727 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:08.373727 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:08.373727 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:08.373727 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:08.373727 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:08.373727 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:08.373727 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:08.373727 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:08.373727 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:08.374641 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:08.374641 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:08.374641 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 02 19:01:08.374641 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 02 19:01:08.374641 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 02 19:01:08.374641 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 02 19:01:08.374641 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 02 19:01:08.374641 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 02 19:01:08.374641 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 02 19:01:08.374641 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:01:08.381397 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 401/809] 10.222.0.33 () {56 vars in 1093 bytes} [Thu Jun 2 19:01:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:08.382299 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28329f26-23e5-407f-8b89-04d81ceb6cff None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:08.473929 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c0b192-d832-4a39-a1d1-ca6881ecab6d None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:08.488667 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28329f26-23e5-407f-8b89-04d81ceb6cff None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:08.513621 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de7ffd6-af7a-4e01-9431-e19528932c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:08.514955 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de7ffd6-af7a-4e01-9431-e19528932c2c None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:08.515442 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de7ffd6-af7a-4e01-9431-e19528932c2c None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:08.515913 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de7ffd6-af7a-4e01-9431-e19528932c2c None admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:08.517048 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de7ffd6-af7a-4e01-9431-e19528932c2c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:08.517592 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de7ffd6-af7a-4e01-9431-e19528932c2c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:08.519421 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de7ffd6-af7a-4e01-9431-e19528932c2c None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:08.525798 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1959353-9250-4ad8-b7fd-20776084cfc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:08.526481 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1959353-9250-4ad8-b7fd-20776084cfc6 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:08.526777 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1959353-9250-4ad8-b7fd-20776084cfc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:08.527071 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1959353-9250-4ad8-b7fd-20776084cfc6 None admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:08.527950 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1959353-9250-4ad8-b7fd-20776084cfc6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:08.528230 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1959353-9250-4ad8-b7fd-20776084cfc6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:08.529682 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1959353-9250-4ad8-b7fd-20776084cfc6 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:08.540910 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 409/810] 10.222.0.33 () {56 vars in 1149 bytes} [Thu Jun 2 19:01:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:08.542163 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de7ffd6-af7a-4e01-9431-e19528932c2c None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:08.550516 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 402/811] 10.222.0.33 () {56 vars in 1133 bytes} [Thu Jun 2 19:01:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:08.551532 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1959353-9250-4ad8-b7fd-20776084cfc6 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:08.585131 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:08.636503 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:08.669443 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de7ffd6-af7a-4e01-9431-e19528932c2c None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:08.681836 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1959353-9250-4ad8-b7fd-20776084cfc6 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:08.709106 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7de6a9-0351-4c89-8238-a3efc01abbe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:08.710014 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7de6a9-0351-4c89-8238-a3efc01abbe4 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:08.710365 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7de6a9-0351-4c89-8238-a3efc01abbe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:08.710708 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7de6a9-0351-4c89-8238-a3efc01abbe4 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:08.712043 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7de6a9-0351-4c89-8238-a3efc01abbe4 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:08.713566 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7de6a9-0351-4c89-8238-a3efc01abbe4 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:08.726353 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-df7de6a9-0351-4c89-8238-a3efc01abbe4 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 02 19:01:08.726353 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:08.726353 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 02 19:01:08.726353 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 02 19:01:08.726353 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 02 19:01:08.726353 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 02 19:01:08.726353 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 02 19:01:08.726353 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 02 19:01:08.726353 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 02 19:01:08.726353 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:08.726353 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 02 19:01:08.726353 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:08.726353 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:08.726353 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:08.726353 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:08.726353 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:08.728115 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:08.728115 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:08.728115 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:08.728115 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:08.728115 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:08.728115 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:08.728115 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:08.728115 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 02 19:01:08.728115 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 02 19:01:08.728115 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 02 19:01:08.728115 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 02 19:01:08.728115 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:08.728115 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:08.728115 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:08.728115 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:08.728115 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:08.729119 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:08.729119 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:08.729119 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:08.729119 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:08.729119 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:08.729119 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:08.729119 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:08.729119 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:08.729119 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:08.729119 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:08.729119 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:08.729119 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 02 19:01:08.729119 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 02 19:01:08.729119 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 02 19:01:08.729119 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:08.732885 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265a887d-42ad-47cb-98b2-39d3d8e1aed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:08.733847 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 410/812] 10.222.0.33 () {56 vars in 1129 bytes} [Thu Jun 2 19:01:08 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:08.734421 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a887d-42ad-47cb-98b2-39d3d8e1aed9 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:08.734814 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7de6a9-0351-4c89-8238-a3efc01abbe4 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:08.735063 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a887d-42ad-47cb-98b2-39d3d8e1aed9 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:08.735557 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a887d-42ad-47cb-98b2-39d3d8e1aed9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:08.751694 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265a887d-42ad-47cb-98b2-39d3d8e1aed9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:08.754344 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265a887d-42ad-47cb-98b2-39d3d8e1aed9 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:08.782110 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-265a887d-42ad-47cb-98b2-39d3d8e1aed9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:01:08.782110 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:08.782110 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:08.782110 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:08.782110 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:08.782110 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:08.782110 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:08.782110 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:08.782110 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:08.782110 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:08.782110 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:08.782110 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:08.782110 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:01:08.782110 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:01:08.782110 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:01:08.783279 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:01:08.783279 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:08.783279 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:08.783279 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:08.783279 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:08.783279 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:08.783279 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:08.783279 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:08.783279 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:08.783279 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:08.783279 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:08.783279 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:08.783279 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:08.783279 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:08.783279 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:08.783279 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:08.783279 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:08.784381 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:01:08.784381 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:01:08.784381 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:01:08.784381 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:01:08.791471 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 403/813] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:01:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:08.791813 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265a887d-42ad-47cb-98b2-39d3d8e1aed9 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:08.859479 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7de6a9-0351-4c89-8238-a3efc01abbe4 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:08.900010 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a859c0-ab31-4170-b4f1-11c41b0557d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:08.901434 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a859c0-ab31-4170-b4f1-11c41b0557d4 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:08.902188 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a859c0-ab31-4170-b4f1-11c41b0557d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:08.902671 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a859c0-ab31-4170-b4f1-11c41b0557d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:08.906908 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:08.907544 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:08.925169 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265a887d-42ad-47cb-98b2-39d3d8e1aed9 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:08.962472 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af339ea0-c17f-4afd-890c-7a72c2afe4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:08.963300 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af339ea0-c17f-4afd-890c-7a72c2afe4d5 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:08.963652 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af339ea0-c17f-4afd-890c-7a72c2afe4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:08.964001 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af339ea0-c17f-4afd-890c-7a72c2afe4d5 None admin] PATH_INFO: `/v3/services` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:08.964794 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af339ea0-c17f-4afd-890c-7a72c2afe4d5 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:08.965173 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af339ea0-c17f-4afd-890c-7a72c2afe4d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:08.966978 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af339ea0-c17f-4afd-890c-7a72c2afe4d5 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:08.977903 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 404/814] 10.222.0.33 () {56 vars in 1124 bytes} [Thu Jun 2 19:01:08 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:08.979587 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af339ea0-c17f-4afd-890c-7a72c2afe4d5 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:09.001228 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a859c0-ab31-4170-b4f1-11c41b0557d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:09.003861 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a859c0-ab31-4170-b4f1-11c41b0557d4 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:09.088526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af339ea0-c17f-4afd-890c-7a72c2afe4d5 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:09.131482 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc932f6-1835-47d1-96f0-b5599fd0ae2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:09.132159 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc932f6-1835-47d1-96f0-b5599fd0ae2e None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:09.132446 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc932f6-1835-47d1-96f0-b5599fd0ae2e None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:09.132725 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc932f6-1835-47d1-96f0-b5599fd0ae2e None admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:09.137528 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 411/815] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 2 19:01:08 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 02 19:01:09.137671 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc932f6-1835-47d1-96f0-b5599fd0ae2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:09.137671 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc932f6-1835-47d1-96f0-b5599fd0ae2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:09.137854 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a859c0-ab31-4170-b4f1-11c41b0557d4 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:09.137938 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc932f6-1835-47d1-96f0-b5599fd0ae2e None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:09.149934 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 405/816] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:01:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:09.151463 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3512153-0121-4900-a491-ff7eb310f9d9 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:09.151754 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3512153-0121-4900-a491-ff7eb310f9d9 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:09.152100 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3512153-0121-4900-a491-ff7eb310f9d9 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:09.153555 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 406/817] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:01:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:01:09.158099 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3512153-0121-4900-a491-ff7eb310f9d9 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:09.170814 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:09.176324 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:09.248479 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a859c0-ab31-4170-b4f1-11c41b0557d4 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:09.273362 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3512153-0121-4900-a491-ff7eb310f9d9 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:09.285227 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5ac621-fa3d-45a5-881a-a5db612d5613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:09.285890 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ac621-fa3d-45a5-881a-a5db612d5613 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:09.286172 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ac621-fa3d-45a5-881a-a5db612d5613 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:09.286442 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ac621-fa3d-45a5-881a-a5db612d5613 None admin] PATH_INFO: `/v3/services` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:09.287181 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5ac621-fa3d-45a5-881a-a5db612d5613 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:09.287486 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5ac621-fa3d-45a5-881a-a5db612d5613 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:09.288910 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5ac621-fa3d-45a5-881a-a5db612d5613 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:09.298430 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 412/818] 10.222.0.33 () {56 vars in 1124 bytes} [Thu Jun 2 19:01:09 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:09.298430 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5ac621-fa3d-45a5-881a-a5db612d5613 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:09.316117 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d29de7-cb29-4c1c-8335-df7897b8c78b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:09.316929 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d29de7-cb29-4c1c-8335-df7897b8c78b None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:09.317225 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d29de7-cb29-4c1c-8335-df7897b8c78b None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:09.317501 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d29de7-cb29-4c1c-8335-df7897b8c78b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:09.329460 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d29de7-cb29-4c1c-8335-df7897b8c78b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:09.330967 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d29de7-cb29-4c1c-8335-df7897b8c78b None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:09.348660 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-74d29de7-cb29-4c1c-8335-df7897b8c78b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:01:09.348660 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:09.348660 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:09.348660 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:09.348660 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:09.348660 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:09.348660 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:09.348660 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:09.348660 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:09.348660 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:09.348660 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:09.348660 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:09.348660 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:01:09.348660 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:01:09.348660 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:01:09.349657 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:01:09.349657 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:09.349657 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:09.349657 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:09.349657 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:09.349657 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:09.349657 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:09.349657 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:09.349657 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:09.349657 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:09.349657 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:09.349657 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:09.349657 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:09.349657 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:09.349657 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:09.349657 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:09.349657 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:09.350540 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:01:09.350540 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:01:09.350540 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:01:09.350540 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:01:09.350540 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 407/819] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:01:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:09.351461 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d8cbd9-3339-4716-98f2-1fa21fba4c7c None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:09.351461 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d8cbd9-3339-4716-98f2-1fa21fba4c7c None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:09.351721 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d8cbd9-3339-4716-98f2-1fa21fba4c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:09.374158 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-e5d8cbd9-3339-4716-98f2-1fa21fba4c7c None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:01:09.375397 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-e5d8cbd9-3339-4716-98f2-1fa21fba4c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:01:09.408726 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:09.412031 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5ac621-fa3d-45a5-881a-a5db612d5613 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:09.436476 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:09.454783 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 408/820] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:01:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 02 19:01:09.454783 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d8cbd9-3339-4716-98f2-1fa21fba4c7c None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:09.461647 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1319b6d8-c487-4036-ae9e-da2d3cc6acca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:09.462760 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1319b6d8-c487-4036-ae9e-da2d3cc6acca None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:09.468240 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1319b6d8-c487-4036-ae9e-da2d3cc6acca None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:09.468240 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1319b6d8-c487-4036-ae9e-da2d3cc6acca None admin] PATH_INFO: `/v3/projects/service` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:09.472865 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1319b6d8-c487-4036-ae9e-da2d3cc6acca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:09.474734 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1319b6d8-c487-4036-ae9e-da2d3cc6acca None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:09.489631 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-1319b6d8-c487-4036-ae9e-da2d3cc6acca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:01:09.489631 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:09.489631 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:09.489631 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:09.489631 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:09.489631 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:09.489631 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:09.489631 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:09.489631 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:09.489631 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:09.489631 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:09.489631 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:09.489631 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 02 19:01:09.489631 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 02 19:01:09.489631 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 02 19:01:09.490592 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 02 19:01:09.490592 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:09.490592 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:09.490592 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:09.490592 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:09.490592 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:09.490592 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:09.490592 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:09.490592 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:09.490592 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:09.490592 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:09.490592 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:09.490592 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:09.490592 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:09.490592 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:09.490592 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:09.490592 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:09.491893 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 02 19:01:09.491893 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 02 19:01:09.491893 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 02 19:01:09.491893 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 02 19:01:09.491893 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 02 19:01:09.491893 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 02 19:01:09.491893 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:01:09.491893 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:09.491893 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 413/821] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 2 19:01:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:09.492533 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1319b6d8-c487-4036-ae9e-da2d3cc6acca None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:09.557056 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d8cbd9-3339-4716-98f2-1fa21fba4c7c None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:09.593752 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5349d3-0bc1-4fe0-9b97-e4bf474a3116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:09.594431 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5349d3-0bc1-4fe0-9b97-e4bf474a3116 None admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:09.594721 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5349d3-0bc1-4fe0-9b97-e4bf474a3116 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:09.595031 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5349d3-0bc1-4fe0-9b97-e4bf474a3116 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:09.595871 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5349d3-0bc1-4fe0-9b97-e4bf474a3116 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:09.597202 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5349d3-0bc1-4fe0-9b97-e4bf474a3116 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:09.605017 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1319b6d8-c487-4036-ae9e-da2d3cc6acca None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:09.640407 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edf7883-9f72-441a-b485-c1eabb1eff3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:09.641076 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edf7883-9f72-441a-b485-c1eabb1eff3f None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:09.641341 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edf7883-9f72-441a-b485-c1eabb1eff3f None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:09.641622 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edf7883-9f72-441a-b485-c1eabb1eff3f None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:09.642347 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edf7883-9f72-441a-b485-c1eabb1eff3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:09.642655 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edf7883-9f72-441a-b485-c1eabb1eff3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:09.644068 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edf7883-9f72-441a-b485-c1eabb1eff3f None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:09.657627 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 414/822] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:01:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:09.658469 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edf7883-9f72-441a-b485-c1eabb1eff3f None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:09.678084 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:09.696319 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 409/823] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Jun 2 19:01:09 2022] POST /identity/v3/endpoints => generated 346 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:01:09.697571 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5349d3-0bc1-4fe0-9b97-e4bf474a3116 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:09.769931 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:09.816172 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edf7883-9f72-441a-b485-c1eabb1eff3f None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:09.869972 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4442f600-3833-4c66-9f65-51c79203ea9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:09.870932 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4442f600-3833-4c66-9f65-51c79203ea9c None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:09.871353 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4442f600-3833-4c66-9f65-51c79203ea9c None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:09.871669 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4442f600-3833-4c66-9f65-51c79203ea9c None admin] PATH_INFO: `/v3/services/network` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:09.872356 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4442f600-3833-4c66-9f65-51c79203ea9c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:09.873797 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4442f600-3833-4c66-9f65-51c79203ea9c None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:09.878088 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5349d3-0bc1-4fe0-9b97-e4bf474a3116 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:09.897028 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-4442f600-3833-4c66-9f65-51c79203ea9c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 02 19:01:09.897028 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:09.897028 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 02 19:01:09.897028 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 02 19:01:09.897028 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 02 19:01:09.897028 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 02 19:01:09.897028 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 02 19:01:09.897028 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 02 19:01:09.897028 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 02 19:01:09.897028 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:09.897028 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 02 19:01:09.897028 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:09.897028 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:09.897028 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:09.897028 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:09.897028 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:09.897028 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:09.898301 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:09.898301 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:09.898301 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:09.898301 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:09.898301 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:09.898301 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:09.898301 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 02 19:01:09.898301 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 02 19:01:09.898301 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 02 19:01:09.898301 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 02 19:01:09.898301 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:09.898301 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:09.898301 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:09.898301 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:09.898301 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:09.898301 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:09.900013 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:09.900013 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:09.900013 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:09.900013 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:09.900013 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:09.900013 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:09.900013 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:09.900013 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:09.900013 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:09.900013 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:09.900013 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 02 19:01:09.900013 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 02 19:01:09.900013 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 02 19:01:09.900013 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:09.907365 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 415/824] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 2 19:01:09 2022] GET /identity/v3/services/network => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:09.907365 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4442f600-3833-4c66-9f65-51c79203ea9c None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:09.935703 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c328553b-1d8a-48c6-9420-339d7e6ebdfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:09.936822 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c328553b-1d8a-48c6-9420-339d7e6ebdfc None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:09.937716 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c328553b-1d8a-48c6-9420-339d7e6ebdfc None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:09.938198 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c328553b-1d8a-48c6-9420-339d7e6ebdfc None admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:09.939319 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c328553b-1d8a-48c6-9420-339d7e6ebdfc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:09.939814 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c328553b-1d8a-48c6-9420-339d7e6ebdfc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:09.942087 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c328553b-1d8a-48c6-9420-339d7e6ebdfc None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:09.958528 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 410/825] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 2 19:01:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:09.960628 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c328553b-1d8a-48c6-9420-339d7e6ebdfc None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:10.025729 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:10.051801 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4442f600-3833-4c66-9f65-51c79203ea9c None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:10.088964 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb01d32-3d4f-4e9b-a2a6-111bc63b3090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:10.089767 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb01d32-3d4f-4e9b-a2a6-111bc63b3090 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:10.090052 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb01d32-3d4f-4e9b-a2a6-111bc63b3090 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:10.090330 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb01d32-3d4f-4e9b-a2a6-111bc63b3090 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:10.097199 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c328553b-1d8a-48c6-9420-339d7e6ebdfc None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:10.097537 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb01d32-3d4f-4e9b-a2a6-111bc63b3090 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:10.099039 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb01d32-3d4f-4e9b-a2a6-111bc63b3090 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:10.099185 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:10.118908 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-0eb01d32-3d4f-4e9b-a2a6-111bc63b3090 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:01:10.118908 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:10.118908 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:10.118908 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:10.118908 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:10.118908 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:10.118908 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:10.118908 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:10.118908 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:10.118908 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:10.118908 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:10.118908 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:10.118908 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 02 19:01:10.118908 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 02 19:01:10.118908 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 02 19:01:10.120029 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 02 19:01:10.120029 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:10.120029 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:10.120029 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:10.120029 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:10.120029 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:10.120029 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:10.120029 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:10.120029 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:10.120029 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:10.120029 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:10.120029 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:10.120029 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:10.120029 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:10.120029 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:10.120029 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:10.120029 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:10.120977 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 02 19:01:10.120977 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 02 19:01:10.120977 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 02 19:01:10.120977 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 02 19:01:10.120977 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 02 19:01:10.120977 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 02 19:01:10.120977 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:01:10.120977 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:10.120977 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 416/826] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 2 19:01:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:10.120977 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb01d32-3d4f-4e9b-a2a6-111bc63b3090 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:10.144057 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311eb709-5b02-41bf-9b6a-4dfd8db998b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:10.144733 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311eb709-5b02-41bf-9b6a-4dfd8db998b2 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:10.145026 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311eb709-5b02-41bf-9b6a-4dfd8db998b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:10.145310 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311eb709-5b02-41bf-9b6a-4dfd8db998b2 None admin] PATH_INFO: `/v3/services` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:10.146032 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311eb709-5b02-41bf-9b6a-4dfd8db998b2 None admin] RBAC: Adding query filter params (name=network) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:10.146346 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311eb709-5b02-41bf-9b6a-4dfd8db998b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:10.147806 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311eb709-5b02-41bf-9b6a-4dfd8db998b2 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:10.153772 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 411/827] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Jun 2 19:01:09 2022] GET /identity/v3/services?name=network => generated 124 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:10.154658 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311eb709-5b02-41bf-9b6a-4dfd8db998b2 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:10.259935 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb01d32-3d4f-4e9b-a2a6-111bc63b3090 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:10.295638 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311eb709-5b02-41bf-9b6a-4dfd8db998b2 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:10.308765 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874ad928-e92f-41ce-b837-c41049cf0d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:10.309801 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ad928-e92f-41ce-b837-c41049cf0d65 None admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:10.310094 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ad928-e92f-41ce-b837-c41049cf0d65 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:10.310639 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ad928-e92f-41ce-b837-c41049cf0d65 None admin] PATH_INFO: `/v3/projects/8c04020eaebd4647804cf8653f32d9e5/users/04f64813f09348eb91cb7bb2869f7612/roles/0e56fba112f94ebba363ffab7f4aa9fe` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:10.322359 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:10.333017 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874ad928-e92f-41ce-b837-c41049cf0d65 None admin] RBAC: Authorizing `identity:create_grant(project_id=8c04020eaebd4647804cf8653f32d9e5, user_id=04f64813f09348eb91cb7bb2869f7612, role_id=0e56fba112f94ebba363ffab7f4aa9fe)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:10.333204 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ceb797-680c-4ac4-99c8-4a35551203ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:10.336828 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ceb797-680c-4ac4-99c8-4a35551203ee None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:10.336828 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ceb797-680c-4ac4-99c8-4a35551203ee None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:10.336828 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ceb797-680c-4ac4-99c8-4a35551203ee None admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:10.336828 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ceb797-680c-4ac4-99c8-4a35551203ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:10.337187 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874ad928-e92f-41ce-b837-c41049cf0d65 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:10.337274 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ceb797-680c-4ac4-99c8-4a35551203ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:10.337274 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ceb797-680c-4ac4-99c8-4a35551203ee None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:10.350706 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 412/828] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 2 19:01:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:10.351480 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ceb797-680c-4ac4-99c8-4a35551203ee None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:10.380706 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:10.423080 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 417/829] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 2 19:01:10 2022] PUT /identity/v3/projects/8c04020eaebd4647804cf8653f32d9e5/users/04f64813f09348eb91cb7bb2869f7612/roles/0e56fba112f94ebba363ffab7f4aa9fe => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:01:10.424117 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874ad928-e92f-41ce-b837-c41049cf0d65 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:10.461087 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ceb797-680c-4ac4-99c8-4a35551203ee None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:10.503703 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196408a8-1721-43b4-92e2-eac9b79d156e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:10.505243 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196408a8-1721-43b4-92e2-eac9b79d156e None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:10.505243 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196408a8-1721-43b4-92e2-eac9b79d156e None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:10.505685 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196408a8-1721-43b4-92e2-eac9b79d156e None admin] PATH_INFO: `/v3/services` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:10.506543 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196408a8-1721-43b4-92e2-eac9b79d156e None admin] RBAC: Adding query filter params (type=network) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:10.507033 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196408a8-1721-43b4-92e2-eac9b79d156e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:10.509263 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196408a8-1721-43b4-92e2-eac9b79d156e None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:10.515668 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 413/830] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Jun 2 19:01:10 2022] GET /identity/v3/services?type=network => generated 350 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:10.517262 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a924fc8-ef3b-4155-8de7-a9a1c9518b39 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:10.517663 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a924fc8-ef3b-4155-8de7-a9a1c9518b39 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:10.518045 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a924fc8-ef3b-4155-8de7-a9a1c9518b39 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:10.524224 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 414/831] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:01:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:01:10.524224 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a924fc8-ef3b-4155-8de7-a9a1c9518b39 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:10.586888 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874ad928-e92f-41ce-b837-c41049cf0d65 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:10.632252 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c0acbb-375f-4664-acdb-37ef0e866b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:10.633950 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c0acbb-375f-4664-acdb-37ef0e866b8e None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:10.634349 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c0acbb-375f-4664-acdb-37ef0e866b8e None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:10.634736 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c0acbb-375f-4664-acdb-37ef0e866b8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:10.636464 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c0acbb-375f-4664-acdb-37ef0e866b8e None admin] RBAC: Adding query filter params (role.id=f13fdd0847924014abf1e95594195e78, scope.project.id=8c04020eaebd4647804cf8653f32d9e5, user.id=70664d5cbdee47ee89c120761939cc75) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:10.637281 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c0acbb-375f-4664-acdb-37ef0e866b8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:10.639530 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c0acbb-375f-4664-acdb-37ef0e866b8e None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:10.650228 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 418/832] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 2 19:01:10 2022] GET /identity/v3/role_assignments?user.id=70664d5cbdee47ee89c120761939cc75&scope.project.id=8c04020eaebd4647804cf8653f32d9e5&role.id=f13fdd0847924014abf1e95594195e78 => generated 608 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:10.652050 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174bb170-988c-4227-acae-2c2d32b7366b None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:10.652459 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174bb170-988c-4227-acae-2c2d32b7366b None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:10.652840 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174bb170-988c-4227-acae-2c2d32b7366b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:10.676376 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:10.683814 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-174bb170-988c-4227-acae-2c2d32b7366b None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:01:10.685426 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-174bb170-988c-4227-acae-2c2d32b7366b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:01:10.692153 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a924fc8-ef3b-4155-8de7-a9a1c9518b39 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:10.741281 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cde268f-80c8-4125-badd-e612a8bc35c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:10.741626 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cde268f-80c8-4125-badd-e612a8bc35c1 None admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:10.741916 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cde268f-80c8-4125-badd-e612a8bc35c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:10.742196 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cde268f-80c8-4125-badd-e612a8bc35c1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:10.743085 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cde268f-80c8-4125-badd-e612a8bc35c1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:10.744778 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cde268f-80c8-4125-badd-e612a8bc35c1 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:10.758247 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:10.792354 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 419/833] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:01:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 02 19:01:10.816536 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174bb170-988c-4227-acae-2c2d32b7366b None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:10.863759 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 415/834] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Jun 2 19:01:10 2022] POST /identity/v3/endpoints => generated 327 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:01:10.963019 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174bb170-988c-4227-acae-2c2d32b7366b None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:11.011813 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaf8229-3c9f-4c12-8ddd-909911c6a8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:11.012648 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaf8229-3c9f-4c12-8ddd-909911c6a8e3 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:11.012934 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaf8229-3c9f-4c12-8ddd-909911c6a8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:11.013211 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaf8229-3c9f-4c12-8ddd-909911c6a8e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:11.026306 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaf8229-3c9f-4c12-8ddd-909911c6a8e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:01:11.034542 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-beaf8229-3c9f-4c12-8ddd-909911c6a8e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 02 19:01:11.034542 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:11.034542 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:11.034542 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:11.034542 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:11.034542 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:11.034542 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:11.034542 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:11.034542 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:11.034542 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:11.034542 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:11.034542 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:11.034542 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:01:11.034542 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:01:11.034542 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:01:11.034542 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:01:11.035582 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:01:11.035582 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:01:11.035582 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:01:11.035582 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:01:11.035582 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:11.035582 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:11.035582 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:11.035582 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:11.035582 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:11.035582 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:11.035582 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:11.035582 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:11.035582 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:11.035582 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:11.035582 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:11.035582 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:11.036408 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:11.036408 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:11.036408 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:11.036408 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:11.036408 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:01:11.036408 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:01:11.036408 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:01:11.036408 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:01:11.036408 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:01:11.036408 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:01:11.036408 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 02 19:01:11.036408 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:11.037035 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 420/835] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 2 19:01:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 02 19:01:11.046831 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cde268f-80c8-4125-badd-e612a8bc35c1 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:11.164176 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cde268f-80c8-4125-badd-e612a8bc35c1 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:11.208025 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c9f199-54a3-47cc-bc2b-9d1dc4379887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:11.208711 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c9f199-54a3-47cc-bc2b-9d1dc4379887 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:11.208998 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c9f199-54a3-47cc-bc2b-9d1dc4379887 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:11.209276 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c9f199-54a3-47cc-bc2b-9d1dc4379887 None admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:11.210035 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c9f199-54a3-47cc-bc2b-9d1dc4379887 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:11.210346 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c9f199-54a3-47cc-bc2b-9d1dc4379887 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:11.211811 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c9f199-54a3-47cc-bc2b-9d1dc4379887 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:11.222094 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 416/836] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 2 19:01:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:11.228240 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaf8229-3c9f-4c12-8ddd-909911c6a8e3 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:11.275642 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:11.347277 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaf8229-3c9f-4c12-8ddd-909911c6a8e3 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:11.383865 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6689b67-72ae-45b2-9e5c-e8c71ef37744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:11.384672 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6689b67-72ae-45b2-9e5c-e8c71ef37744 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:11.384960 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6689b67-72ae-45b2-9e5c-e8c71ef37744 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:11.385244 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6689b67-72ae-45b2-9e5c-e8c71ef37744 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:11.396458 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6689b67-72ae-45b2-9e5c-e8c71ef37744 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:11.397945 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6689b67-72ae-45b2-9e5c-e8c71ef37744 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:11.417304 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-c6689b67-72ae-45b2-9e5c-e8c71ef37744 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:01:11.417304 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:11.417304 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:11.417304 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:11.417304 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:11.417304 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:11.417304 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:11.417304 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:11.417304 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:11.417304 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:11.417304 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:11.417304 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:11.417304 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:01:11.417304 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:01:11.417304 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:01:11.418252 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:01:11.418252 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:11.418252 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:11.418252 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:11.418252 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:11.418252 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:11.418252 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:11.418252 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:11.418252 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:11.418252 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:11.418252 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:11.418252 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:11.418252 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:11.418252 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:11.418252 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:11.418252 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:11.418252 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:11.419177 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:01:11.419177 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:01:11.419177 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:01:11.419177 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:11.419177 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 421/837] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:01:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:11.426039 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c9f199-54a3-47cc-bc2b-9d1dc4379887 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:11.438595 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:11.533726 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c9f199-54a3-47cc-bc2b-9d1dc4379887 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:11.569316 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb56ce05-eb0c-4532-ad94-638b754648c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:11.570332 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb56ce05-eb0c-4532-ad94-638b754648c2 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:11.571443 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb56ce05-eb0c-4532-ad94-638b754648c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:11.571919 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb56ce05-eb0c-4532-ad94-638b754648c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:11.572849 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb56ce05-eb0c-4532-ad94-638b754648c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:11.573351 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb56ce05-eb0c-4532-ad94-638b754648c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:11.574932 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb56ce05-eb0c-4532-ad94-638b754648c2 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:11.588137 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 417/838] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:01:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:11.593880 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6689b67-72ae-45b2-9e5c-e8c71ef37744 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:11.687303 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:11.697115 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6689b67-72ae-45b2-9e5c-e8c71ef37744 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:11.731991 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87505ad6-1b96-4feb-86b9-15ce1066d0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:11.732804 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87505ad6-1b96-4feb-86b9-15ce1066d0d1 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:11.733085 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87505ad6-1b96-4feb-86b9-15ce1066d0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:11.733357 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87505ad6-1b96-4feb-86b9-15ce1066d0d1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:11.741612 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87505ad6-1b96-4feb-86b9-15ce1066d0d1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:11.743198 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87505ad6-1b96-4feb-86b9-15ce1066d0d1 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:11.758817 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-87505ad6-1b96-4feb-86b9-15ce1066d0d1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 02 19:01:11.758817 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:11.758817 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:11.758817 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:11.758817 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:11.758817 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:11.758817 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:11.758817 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:11.758817 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:11.758817 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:11.758817 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:11.758817 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:11.758817 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 02 19:01:11.758817 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 02 19:01:11.758817 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 02 19:01:11.758817 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 02 19:01:11.759815 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:11.759815 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:11.759815 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 02 19:01:11.759815 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:01:11.759815 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 02 19:01:11.759815 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:01:11.759815 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:11.759815 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:11.759815 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:11.759815 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:11.759815 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:11.759815 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:11.759815 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:11.759815 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:11.759815 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:11.759815 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:11.759815 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:11.759815 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:11.760743 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:11.760743 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:11.760743 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 02 19:01:11.760743 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 02 19:01:11.760743 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 02 19:01:11.760743 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 02 19:01:11.760743 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 02 19:01:11.760743 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 02 19:01:11.760743 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 02 19:01:11.760743 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:11.760743 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 422/839] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 2 19:01:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:11.766024 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb56ce05-eb0c-4532-ad94-638b754648c2 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:11.846501 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:11.882024 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb56ce05-eb0c-4532-ad94-638b754648c2 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:11.923292 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64182660-ac57-4abe-9bfa-5a0e18440585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:11.924085 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64182660-ac57-4abe-9bfa-5a0e18440585 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:11.924369 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64182660-ac57-4abe-9bfa-5a0e18440585 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:11.924642 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64182660-ac57-4abe-9bfa-5a0e18440585 None admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:11.925495 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64182660-ac57-4abe-9bfa-5a0e18440585 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:11.925801 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64182660-ac57-4abe-9bfa-5a0e18440585 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:11.927287 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64182660-ac57-4abe-9bfa-5a0e18440585 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:11.949934 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 418/840] 10.222.0.33 () {56 vars in 1135 bytes} [Thu Jun 2 19:01:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:11.955352 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87505ad6-1b96-4feb-86b9-15ce1066d0d1 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:12.061613 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87505ad6-1b96-4feb-86b9-15ce1066d0d1 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:12.097000 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14e55ab-c394-48bf-ab06-5c0f7817efb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:12.097994 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e55ab-c394-48bf-ab06-5c0f7817efb3 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:12.098503 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e55ab-c394-48bf-ab06-5c0f7817efb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:12.099043 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e55ab-c394-48bf-ab06-5c0f7817efb3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:12.128124 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:12.182574 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14e55ab-c394-48bf-ab06-5c0f7817efb3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:12.184334 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14e55ab-c394-48bf-ab06-5c0f7817efb3 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:12.242203 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d71cf8-5a2f-434b-9731-6a40b80c9725 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:12.242570 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d71cf8-5a2f-434b-9731-6a40b80c9725 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:12.242822 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d71cf8-5a2f-434b-9731-6a40b80c9725 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:12.244787 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 419/841] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:01:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 02 19:01:12.250047 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d6f35-6deb-45ae-9743-0c0b6cfd1c37 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:12.250334 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d6f35-6deb-45ae-9743-0c0b6cfd1c37 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:12.250613 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d6f35-6deb-45ae-9743-0c0b6cfd1c37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:12.269585 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-da8d6f35-6deb-45ae-9743-0c0b6cfd1c37 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:01:12.271203 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-da8d6f35-6deb-45ae-9743-0c0b6cfd1c37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:01:12.310826 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 423/842] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 2 19:01:11 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 02 19:01:12.313906 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14e55ab-c394-48bf-ab06-5c0f7817efb3 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:12.361633 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 420/843] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:01:12 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 02 19:01:12.375084 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:12.385307 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8d6f35-6deb-45ae-9743-0c0b6cfd1c37 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:12.430010 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14e55ab-c394-48bf-ab06-5c0f7817efb3 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:12.456207 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:12.475250 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b2d8db-28c8-4b7e-a89f-110f4fb46e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:12.476052 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b2d8db-28c8-4b7e-a89f-110f4fb46e78 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:12.476378 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b2d8db-28c8-4b7e-a89f-110f4fb46e78 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:12.476608 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b2d8db-28c8-4b7e-a89f-110f4fb46e78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:12.495580 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b2d8db-28c8-4b7e-a89f-110f4fb46e78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:12.497235 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b2d8db-28c8-4b7e-a89f-110f4fb46e78 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:12.508051 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8d6f35-6deb-45ae-9743-0c0b6cfd1c37 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:12.521312 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-26b2d8db-28c8-4b7e-a89f-110f4fb46e78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:01:12.521312 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:12.521312 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:12.521312 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:12.521312 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:12.521312 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:12.521312 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:12.521312 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:12.521312 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:12.521312 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:12.521312 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:12.521312 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:12.521312 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:01:12.521312 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:01:12.521312 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:01:12.522273 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:01:12.522273 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:12.522273 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:12.522273 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:12.522273 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:12.522273 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:12.522273 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:12.522273 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:12.522273 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:12.522273 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:12.522273 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:12.522273 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:12.522273 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:12.522273 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:12.522273 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:12.522273 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:12.522273 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:12.523572 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:01:12.523572 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:01:12.523572 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:01:12.523572 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:12.526824 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 424/844] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:01:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:12.529153 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b2d8db-28c8-4b7e-a89f-110f4fb46e78 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:12.552770 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49f7bca-6d85-4a87-8249-3eacc7cc2ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:12.553676 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49f7bca-6d85-4a87-8249-3eacc7cc2ada None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:12.553967 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49f7bca-6d85-4a87-8249-3eacc7cc2ada None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:12.554241 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49f7bca-6d85-4a87-8249-3eacc7cc2ada None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:12.555700 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49f7bca-6d85-4a87-8249-3eacc7cc2ada None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:12.557004 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49f7bca-6d85-4a87-8249-3eacc7cc2ada None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:12.570411 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-d49f7bca-6d85-4a87-8249-3eacc7cc2ada None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 02 19:01:12.570411 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:12.570411 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 02 19:01:12.570411 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 02 19:01:12.570411 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 02 19:01:12.570411 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 02 19:01:12.570411 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 02 19:01:12.570411 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 02 19:01:12.570411 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 02 19:01:12.570411 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:01:12.570411 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 02 19:01:12.570411 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:01:12.570411 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:12.570411 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:12.570411 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:12.570411 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:12.570411 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:12.571659 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:12.571659 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:12.571659 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:12.571659 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:12.571659 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:12.571659 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:12.571659 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 02 19:01:12.571659 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 02 19:01:12.571659 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 02 19:01:12.571659 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 02 19:01:12.571659 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:12.571659 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:12.571659 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:12.571659 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:12.571659 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:12.571659 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:12.572565 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:12.572565 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:12.572565 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:12.572565 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:12.572565 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:12.572565 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:12.572565 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:12.572565 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:12.572565 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:12.572565 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:12.572565 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 02 19:01:12.572565 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 02 19:01:12.572565 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 02 19:01:12.572565 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:01:12.577041 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 421/845] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 2 19:01:12 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 02 19:01:12.577941 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49f7bca-6d85-4a87-8249-3eacc7cc2ada None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:12.674421 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b2d8db-28c8-4b7e-a89f-110f4fb46e78 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:12.676986 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:12.703452 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49f7bca-6d85-4a87-8249-3eacc7cc2ada None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:12.714203 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecfbe12-bac0-4ca9-9ae8-b0bb5e83d73c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:12.715073 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecfbe12-bac0-4ca9-9ae8-b0bb5e83d73c None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:12.715461 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecfbe12-bac0-4ca9-9ae8-b0bb5e83d73c None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:12.715832 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecfbe12-bac0-4ca9-9ae8-b0bb5e83d73c None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:12.716676 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecfbe12-bac0-4ca9-9ae8-b0bb5e83d73c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:12.717092 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecfbe12-bac0-4ca9-9ae8-b0bb5e83d73c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:12.718585 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecfbe12-bac0-4ca9-9ae8-b0bb5e83d73c None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:12.729905 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:12.735770 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 425/846] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:01:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:12.737263 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecfbe12-bac0-4ca9-9ae8-b0bb5e83d73c None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:12.753439 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd386b2e-e808-4078-81fe-304cb4ee895a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:12.754801 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd386b2e-e808-4078-81fe-304cb4ee895a None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:12.755257 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd386b2e-e808-4078-81fe-304cb4ee895a None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:12.757103 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd386b2e-e808-4078-81fe-304cb4ee895a None admin] PATH_INFO: `/v3/services` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:12.758172 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd386b2e-e808-4078-81fe-304cb4ee895a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:12.759223 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd386b2e-e808-4078-81fe-304cb4ee895a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:12.761269 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd386b2e-e808-4078-81fe-304cb4ee895a None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:12.767961 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 422/847] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 2 19:01:12 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:12.773403 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd386b2e-e808-4078-81fe-304cb4ee895a None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:12.856344 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecfbe12-bac0-4ca9-9ae8-b0bb5e83d73c None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:12.882640 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd386b2e-e808-4078-81fe-304cb4ee895a None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:12.894077 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fcd282-5764-47b3-835e-6951d3ede46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:12.894923 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fcd282-5764-47b3-835e-6951d3ede46c None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:12.895243 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fcd282-5764-47b3-835e-6951d3ede46c None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:12.895729 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fcd282-5764-47b3-835e-6951d3ede46c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:12.903739 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fcd282-5764-47b3-835e-6951d3ede46c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:12.905271 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fcd282-5764-47b3-835e-6951d3ede46c None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:12.921535 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92457518-b6be-44e9-bec4-8190db3d1e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:12.922554 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92457518-b6be-44e9-bec4-8190db3d1e7e None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:12.922664 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-c3fcd282-5764-47b3-835e-6951d3ede46c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:01:12.922664 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:12.922664 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:12.922664 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:12.922664 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:12.922664 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:12.922664 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:12.922664 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:12.922664 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:12.922664 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:12.922664 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:12.922664 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:12.922664 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 02 19:01:12.922664 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 02 19:01:12.922664 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 02 19:01:12.923629 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 02 19:01:12.923629 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:12.923629 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:12.923629 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:12.923629 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:12.923629 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:12.923629 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:12.923629 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:12.923629 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:12.923629 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:12.923629 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:12.923629 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:12.923629 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:12.923629 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:12.923629 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:12.923629 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:12.923629 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:12.924576 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 02 19:01:12.924576 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 02 19:01:12.924576 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 02 19:01:12.924576 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 02 19:01:12.924576 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 02 19:01:12.924576 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 02 19:01:12.924576 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:01:12.924576 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:12.925741 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92457518-b6be-44e9-bec4-8190db3d1e7e None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:12.926243 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92457518-b6be-44e9-bec4-8190db3d1e7e None admin] PATH_INFO: `/v3/services` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:12.927294 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92457518-b6be-44e9-bec4-8190db3d1e7e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:12.927820 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92457518-b6be-44e9-bec4-8190db3d1e7e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:12.929012 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 426/848] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 2 19:01:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:12.930584 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fcd282-5764-47b3-835e-6951d3ede46c None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:12.930863 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92457518-b6be-44e9-bec4-8190db3d1e7e None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:12.937365 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 423/849] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 2 19:01:12 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:01:12.985466 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad4e153-8ede-4fb4-990a-db47340a7d63 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:12.985666 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:12.985748 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad4e153-8ede-4fb4-990a-db47340a7d63 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:12.986022 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad4e153-8ede-4fb4-990a-db47340a7d63 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:12.991825 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 424/850] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:01:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 02 19:01:12.994422 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75255e14-282c-46ea-a01d-cae43cc37797 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:12.994711 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75255e14-282c-46ea-a01d-cae43cc37797 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:12.997131 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75255e14-282c-46ea-a01d-cae43cc37797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:13.013959 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-75255e14-282c-46ea-a01d-cae43cc37797 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:01:13.015169 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-75255e14-282c-46ea-a01d-cae43cc37797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:01:13.066868 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fcd282-5764-47b3-835e-6951d3ede46c None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:13.076160 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:13.115383 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 425/851] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:01:12 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 02 19:01:13.118744 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3dba70-768d-46d0-8434-2f5328df120b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:13.119724 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3dba70-768d-46d0-8434-2f5328df120b None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:13.120232 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3dba70-768d-46d0-8434-2f5328df120b None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:13.120740 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3dba70-768d-46d0-8434-2f5328df120b None admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:13.124176 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3dba70-768d-46d0-8434-2f5328df120b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:13.124867 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3dba70-768d-46d0-8434-2f5328df120b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:13.126906 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3dba70-768d-46d0-8434-2f5328df120b None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:13.134659 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75255e14-282c-46ea-a01d-cae43cc37797 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:13.142379 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 427/852] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 2 19:01:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:13.151854 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3dba70-768d-46d0-8434-2f5328df120b None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:13.267086 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75255e14-282c-46ea-a01d-cae43cc37797 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:13.270527 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:13.298542 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3dba70-768d-46d0-8434-2f5328df120b None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:13.313944 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0167360-263b-4e38-a2d5-452d29c19e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:13.315121 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0167360-263b-4e38-a2d5-452d29c19e19 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:13.315420 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0167360-263b-4e38-a2d5-452d29c19e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:13.315703 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0167360-263b-4e38-a2d5-452d29c19e19 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:13.332705 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0167360-263b-4e38-a2d5-452d29c19e19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:01:13.340594 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbe1b20-cd41-47eb-be4f-10fb3b08c719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:13.341385 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbe1b20-cd41-47eb-be4f-10fb3b08c719 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:13.341740 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbe1b20-cd41-47eb-be4f-10fb3b08c719 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:13.342094 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbe1b20-cd41-47eb-be4f-10fb3b08c719 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:13.342229 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-f0167360-263b-4e38-a2d5-452d29c19e19 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 02 19:01:13.342229 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:13.342229 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:13.342229 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:13.342229 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:13.342229 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:13.342229 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:13.342229 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:13.342229 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:13.342229 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:13.342229 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:13.342229 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:13.342229 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:01:13.342229 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:01:13.342229 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:01:13.342229 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:01:13.343104 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:01:13.343104 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:01:13.343104 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:01:13.343104 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:01:13.343104 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:13.343104 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:13.343104 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:13.343104 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:13.343104 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:13.343104 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:13.343104 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:13.343104 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:13.343104 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:13.343104 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:13.343104 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:13.343104 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:13.343927 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:13.343927 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:13.343927 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:13.343927 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:13.343927 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:01:13.343927 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:01:13.343927 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:01:13.343927 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:01:13.343927 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:01:13.343927 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:01:13.343927 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 02 19:01:13.343927 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:01:13.344540 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbe1b20-cd41-47eb-be4f-10fb3b08c719 None admin] RBAC: Adding query filter params (role.id=f13fdd0847924014abf1e95594195e78, scope.project.id=8c04020eaebd4647804cf8653f32d9e5, user.id=2a59e1d67ff94186a3ebcfcb49938a87) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:13.344540 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbe1b20-cd41-47eb-be4f-10fb3b08c719 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:13.344663 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 426/853] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 2 19:01:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:13.349660 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbe1b20-cd41-47eb-be4f-10fb3b08c719 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:13.354271 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0167360-263b-4e38-a2d5-452d29c19e19 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:13.357758 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 428/854] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 2 19:01:13 2022] GET /identity/v3/role_assignments?user.id=2a59e1d67ff94186a3ebcfcb49938a87&scope.project.id=8c04020eaebd4647804cf8653f32d9e5&role.id=f13fdd0847924014abf1e95594195e78 => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:13.415649 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:13.437716 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da03413f-1c5e-4db0-95c3-f401f29e015e None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:13.438057 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da03413f-1c5e-4db0-95c3-f401f29e015e None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:13.438287 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da03413f-1c5e-4db0-95c3-f401f29e015e None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:13.440092 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 429/855] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:01:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 02 19:01:13.445983 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742f173d-84c2-4fe3-88f3-facb72bbd10f None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:13.446291 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742f173d-84c2-4fe3-88f3-facb72bbd10f None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:13.446552 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742f173d-84c2-4fe3-88f3-facb72bbd10f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:13.465019 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-742f173d-84c2-4fe3-88f3-facb72bbd10f None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:01:13.466275 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-742f173d-84c2-4fe3-88f3-facb72bbd10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:01:13.492331 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0167360-263b-4e38-a2d5-452d29c19e19 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:13.534302 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f897a-7cfc-40f1-ae35-ddc642587134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:13.534975 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f897a-7cfc-40f1-ae35-ddc642587134 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:13.535263 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f897a-7cfc-40f1-ae35-ddc642587134 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:13.535599 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f897a-7cfc-40f1-ae35-ddc642587134 None admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:13.536332 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415f897a-7cfc-40f1-ae35-ddc642587134 None admin] RBAC: Adding query filter params (name=service) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:13.536648 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415f897a-7cfc-40f1-ae35-ddc642587134 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:13.538099 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415f897a-7cfc-40f1-ae35-ddc642587134 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:13.553051 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 427/856] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 2 19:01:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:01:13.554586 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f897a-7cfc-40f1-ae35-ddc642587134 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:13.568001 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 430/857] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:01:13 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 02 19:01:13.591701 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742f173d-84c2-4fe3-88f3-facb72bbd10f None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:13.693034 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:13.697387 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f897a-7cfc-40f1-ae35-ddc642587134 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:13.702485 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:13.741833 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d314205-1df7-43d5-ac12-de886ad279f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:13.743553 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d314205-1df7-43d5-ac12-de886ad279f0 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:13.743987 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d314205-1df7-43d5-ac12-de886ad279f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:13.744577 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d314205-1df7-43d5-ac12-de886ad279f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:13.753001 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742f173d-84c2-4fe3-88f3-facb72bbd10f None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:13.765465 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d314205-1df7-43d5-ac12-de886ad279f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:13.767353 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d314205-1df7-43d5-ac12-de886ad279f0 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:13.792220 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-4d314205-1df7-43d5-ac12-de886ad279f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:01:13.792220 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:13.792220 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:13.792220 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:13.792220 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:13.792220 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:13.792220 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:13.792220 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:13.792220 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:13.792220 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:13.792220 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:13.792220 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:13.792220 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:01:13.792220 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:01:13.792220 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:01:13.793328 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:01:13.793328 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:13.793328 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:13.793328 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:13.793328 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:13.793328 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:13.793328 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:13.793328 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:13.793328 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:13.793328 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:13.793328 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:13.793328 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:13.793328 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:13.793328 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:13.793328 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:13.793328 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:13.793328 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:13.795452 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:01:13.795452 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:01:13.795452 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:01:13.795452 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:01:13.795452 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 428/858] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:01:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:13.799554 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d314205-1df7-43d5-ac12-de886ad279f0 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:13.800559 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6479a0c-7b38-4f41-8ca5-d6d6d2b80771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:13.801740 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6479a0c-7b38-4f41-8ca5-d6d6d2b80771 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:13.802270 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6479a0c-7b38-4f41-8ca5-d6d6d2b80771 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:13.802742 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6479a0c-7b38-4f41-8ca5-d6d6d2b80771 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:13.803919 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6479a0c-7b38-4f41-8ca5-d6d6d2b80771 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:13.805942 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6479a0c-7b38-4f41-8ca5-d6d6d2b80771 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:13.819213 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-a6479a0c-7b38-4f41-8ca5-d6d6d2b80771 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 02 19:01:13.819213 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:13.819213 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 02 19:01:13.819213 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 02 19:01:13.819213 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 02 19:01:13.819213 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 02 19:01:13.819213 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 02 19:01:13.819213 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 02 19:01:13.819213 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 02 19:01:13.819213 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:13.819213 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 02 19:01:13.819213 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:13.819213 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:13.819213 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:13.819213 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:13.819213 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:13.820392 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:13.820392 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:13.820392 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:13.820392 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:13.820392 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:13.820392 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:13.820392 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:13.820392 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 02 19:01:13.820392 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 02 19:01:13.820392 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 02 19:01:13.820392 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 02 19:01:13.820392 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:13.820392 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:13.820392 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:13.820392 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:13.820392 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:13.821873 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:13.821873 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:13.821873 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:13.821873 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:13.821873 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:13.821873 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:13.821873 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:13.821873 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:13.821873 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:13.821873 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:13.821873 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:13.821873 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 02 19:01:13.821873 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 02 19:01:13.821873 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 02 19:01:13.821873 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:13.822644 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 431/859] 10.222.0.33 () {56 vars in 1132 bytes} [Thu Jun 2 19:01:13 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 02 19:01:13.827702 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6479a0c-7b38-4f41-8ca5-d6d6d2b80771 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:13.940037 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d314205-1df7-43d5-ac12-de886ad279f0 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:13.953572 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6479a0c-7b38-4f41-8ca5-d6d6d2b80771 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:13.979758 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187a36b3-b518-4a32-b25b-ed7d599db691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:13.980461 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a36b3-b518-4a32-b25b-ed7d599db691 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:13.980748 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a36b3-b518-4a32-b25b-ed7d599db691 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:13.981034 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a36b3-b518-4a32-b25b-ed7d599db691 None admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:13.981761 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187a36b3-b518-4a32-b25b-ed7d599db691 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:13.982071 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187a36b3-b518-4a32-b25b-ed7d599db691 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:13.983805 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187a36b3-b518-4a32-b25b-ed7d599db691 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:13.991669 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8eb826-a5c6-4048-9d98-5f235d0a0f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:13.992361 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8eb826-a5c6-4048-9d98-5f235d0a0f8d None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:13.992678 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8eb826-a5c6-4048-9d98-5f235d0a0f8d None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:13.992961 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8eb826-a5c6-4048-9d98-5f235d0a0f8d None admin] PATH_INFO: `/v3/services` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:13.993690 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8eb826-a5c6-4048-9d98-5f235d0a0f8d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:13.994001 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8eb826-a5c6-4048-9d98-5f235d0a0f8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:13.995478 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8eb826-a5c6-4048-9d98-5f235d0a0f8d None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:14.003558 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 429/860] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:01:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:14.005334 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187a36b3-b518-4a32-b25b-ed7d599db691 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:14.005519 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 432/861] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Jun 2 19:01:13 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:14.006526 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8eb826-a5c6-4048-9d98-5f235d0a0f8d None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:14.011241 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:14.023555 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:14.115227 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8eb826-a5c6-4048-9d98-5f235d0a0f8d None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:14.121803 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187a36b3-b518-4a32-b25b-ed7d599db691 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:14.158185 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8affcb3-5e87-4d56-b427-746262f11dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:14.158937 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8affcb3-5e87-4d56-b427-746262f11dad None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:14.159230 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8affcb3-5e87-4d56-b427-746262f11dad None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:14.159519 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8affcb3-5e87-4d56-b427-746262f11dad None admin] PATH_INFO: `/v3/services` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:14.160409 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8affcb3-5e87-4d56-b427-746262f11dad None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:14.160721 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8affcb3-5e87-4d56-b427-746262f11dad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:14.161489 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b32865-f480-4a89-bc08-03a930464f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:14.162105 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8affcb3-5e87-4d56-b427-746262f11dad None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:14.162625 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b32865-f480-4a89-bc08-03a930464f11 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:14.163088 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b32865-f480-4a89-bc08-03a930464f11 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:14.163604 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b32865-f480-4a89-bc08-03a930464f11 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:14.173292 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b32865-f480-4a89-bc08-03a930464f11 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:14.175091 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b32865-f480-4a89-bc08-03a930464f11 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:14.175286 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 433/862] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Jun 2 19:01:14 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:14.192531 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-35b32865-f480-4a89-bc08-03a930464f11 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 02 19:01:14.192531 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:14.192531 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:14.192531 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:14.192531 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:14.192531 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:14.192531 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:14.192531 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:14.192531 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:14.192531 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:14.192531 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:14.192531 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:14.192531 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 02 19:01:14.192531 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 02 19:01:14.192531 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 02 19:01:14.192531 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 02 19:01:14.193708 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:14.193708 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:14.193708 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 02 19:01:14.193708 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:01:14.193708 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 02 19:01:14.193708 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:01:14.193708 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:14.193708 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:14.193708 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:14.193708 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:14.193708 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:14.193708 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:14.193708 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:14.193708 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:14.193708 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:14.193708 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:14.193708 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:14.193708 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:14.194734 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:14.194734 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:14.194734 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 02 19:01:14.194734 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 02 19:01:14.194734 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 02 19:01:14.194734 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 02 19:01:14.194734 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 02 19:01:14.194734 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 02 19:01:14.194734 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 02 19:01:14.194734 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:01:14.201635 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 430/863] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Jun 2 19:01:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:14.203349 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8affcb3-5e87-4d56-b427-746262f11dad None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:14.280406 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:14.322224 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8affcb3-5e87-4d56-b427-746262f11dad None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:14.360857 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab710d65-f3ed-4116-8e12-5687b1e9197f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:14.361903 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab710d65-f3ed-4116-8e12-5687b1e9197f None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:14.362337 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab710d65-f3ed-4116-8e12-5687b1e9197f None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:14.362785 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab710d65-f3ed-4116-8e12-5687b1e9197f None admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:14.363791 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab710d65-f3ed-4116-8e12-5687b1e9197f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:14.364265 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab710d65-f3ed-4116-8e12-5687b1e9197f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:14.365975 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab710d65-f3ed-4116-8e12-5687b1e9197f None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:14.395936 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 434/864] 10.222.0.33 () {56 vars in 1149 bytes} [Thu Jun 2 19:01:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:14.402887 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b32865-f480-4a89-bc08-03a930464f11 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:14.486006 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:14.520302 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b32865-f480-4a89-bc08-03a930464f11 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:14.557066 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d23638-e2c8-478e-aeed-692cb5584e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:14.558010 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d23638-e2c8-478e-aeed-692cb5584e59 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:14.558444 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d23638-e2c8-478e-aeed-692cb5584e59 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:14.558901 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d23638-e2c8-478e-aeed-692cb5584e59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:14.630739 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d23638-e2c8-478e-aeed-692cb5584e59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:14.632470 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d23638-e2c8-478e-aeed-692cb5584e59 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:14.703890 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:14.738869 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 431/865] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 2 19:01:14 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 02 19:01:14.740972 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab710d65-f3ed-4116-8e12-5687b1e9197f None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:14.912791 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab710d65-f3ed-4116-8e12-5687b1e9197f None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:14.963360 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:14.977355 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0cafc9-3b41-4810-8929-9a3cf7520721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:14.982731 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0cafc9-3b41-4810-8929-9a3cf7520721 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:14.982731 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0cafc9-3b41-4810-8929-9a3cf7520721 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:14.982731 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0cafc9-3b41-4810-8929-9a3cf7520721 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:14.990455 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0cafc9-3b41-4810-8929-9a3cf7520721 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:14.992229 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0cafc9-3b41-4810-8929-9a3cf7520721 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:15.013760 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-af0cafc9-3b41-4810-8929-9a3cf7520721 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:01:15.013760 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:15.013760 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:15.013760 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:15.013760 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:15.013760 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:15.013760 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:15.013760 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:15.013760 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:15.013760 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:15.013760 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:15.013760 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:15.013760 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:01:15.013760 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:01:15.013760 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:01:15.014909 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:01:15.014909 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:15.014909 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:15.014909 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:15.014909 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:15.014909 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:15.014909 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:15.014909 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:15.014909 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:15.014909 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:15.014909 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:15.014909 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:15.014909 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:15.014909 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:15.014909 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:15.014909 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:15.014909 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:15.015986 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:01:15.015986 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:01:15.015986 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:01:15.015986 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:15.017829 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 435/866] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:01:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:15.023229 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d23638-e2c8-478e-aeed-692cb5584e59 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:15.126554 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d23638-e2c8-478e-aeed-692cb5584e59 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:15.162984 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabd7d0f-1616-45fb-a2c7-b91a3a63f33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:15.164033 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd7d0f-1616-45fb-a2c7-b91a3a63f33e None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:15.164391 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd7d0f-1616-45fb-a2c7-b91a3a63f33e None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:15.165252 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd7d0f-1616-45fb-a2c7-b91a3a63f33e None admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:15.165976 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabd7d0f-1616-45fb-a2c7-b91a3a63f33e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:15.166287 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabd7d0f-1616-45fb-a2c7-b91a3a63f33e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:15.167690 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabd7d0f-1616-45fb-a2c7-b91a3a63f33e None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:15.184126 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 432/867] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:01:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:15.185590 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0cafc9-3b41-4810-8929-9a3cf7520721 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:15.316250 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0cafc9-3b41-4810-8929-9a3cf7520721 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:15.381613 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb63e94f-e514-41e2-8409-b8f64c3b0376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:15.383064 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63e94f-e514-41e2-8409-b8f64c3b0376 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:15.384107 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63e94f-e514-41e2-8409-b8f64c3b0376 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:15.384626 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63e94f-e514-41e2-8409-b8f64c3b0376 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:15.398332 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb63e94f-e514-41e2-8409-b8f64c3b0376 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:15.401886 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb63e94f-e514-41e2-8409-b8f64c3b0376 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:15.429882 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-eb63e94f-e514-41e2-8409-b8f64c3b0376 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:01:15.429882 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:15.429882 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:15.429882 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:15.429882 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:15.429882 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:15.429882 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:15.429882 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:15.429882 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:15.429882 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:15.429882 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:15.429882 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:15.429882 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 02 19:01:15.429882 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 02 19:01:15.429882 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 02 19:01:15.430883 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 02 19:01:15.430883 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:15.430883 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:15.430883 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:15.430883 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:15.430883 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:15.430883 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:15.430883 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:15.430883 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:15.430883 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:15.430883 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:15.430883 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:15.430883 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:15.430883 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:15.430883 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:15.430883 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:15.430883 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:15.431783 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 02 19:01:15.431783 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 02 19:01:15.431783 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 02 19:01:15.431783 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 02 19:01:15.431783 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 02 19:01:15.431783 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 02 19:01:15.431783 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:01:15.431783 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:15.432527 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 436/868] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 2 19:01:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:15.437893 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabd7d0f-1616-45fb-a2c7-b91a3a63f33e None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:15.482605 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:15.555214 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabd7d0f-1616-45fb-a2c7-b91a3a63f33e None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:15.593553 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff2c75b-8fe1-429f-8979-172a57b848d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:15.594740 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff2c75b-8fe1-429f-8979-172a57b848d2 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:15.595268 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff2c75b-8fe1-429f-8979-172a57b848d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:15.595776 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff2c75b-8fe1-429f-8979-172a57b848d2 None admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:15.596784 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff2c75b-8fe1-429f-8979-172a57b848d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:15.597315 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff2c75b-8fe1-429f-8979-172a57b848d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:15.598987 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff2c75b-8fe1-429f-8979-172a57b848d2 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:15.613511 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 433/869] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 2 19:01:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:15.619989 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb63e94f-e514-41e2-8409-b8f64c3b0376 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:15.668159 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0fe4c3-0583-4249-8b90-0d5d73a11829 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:15.668838 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0fe4c3-0583-4249-8b90-0d5d73a11829 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:15.669372 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0fe4c3-0583-4249-8b90-0d5d73a11829 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:15.671652 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 434/870] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:01:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 02 19:01:15.674597 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:15.683103 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae930e-c6c2-4037-ac7e-132c8b3a86d3 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:15.683103 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae930e-c6c2-4037-ac7e-132c8b3a86d3 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:15.683103 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae930e-c6c2-4037-ac7e-132c8b3a86d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:15.703443 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-5aae930e-c6c2-4037-ac7e-132c8b3a86d3 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:01:15.705318 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-5aae930e-c6c2-4037-ac7e-132c8b3a86d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:01:15.748804 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb63e94f-e514-41e2-8409-b8f64c3b0376 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:15.787561 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 435/871] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:01:15 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 02 19:01:15.796220 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2704bcb-e1e3-4290-a2cc-596fa008b4ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:15.797167 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2704bcb-e1e3-4290-a2cc-596fa008b4ae None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:15.797666 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2704bcb-e1e3-4290-a2cc-596fa008b4ae None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:15.798180 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2704bcb-e1e3-4290-a2cc-596fa008b4ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:15.799317 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2704bcb-e1e3-4290-a2cc-596fa008b4ae None admin] RBAC: Adding query filter params (role.id=0e56fba112f94ebba363ffab7f4aa9fe, scope.project.id=8c04020eaebd4647804cf8653f32d9e5, user.id=04f64813f09348eb91cb7bb2869f7612) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:15.799872 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2704bcb-e1e3-4290-a2cc-596fa008b4ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:15.801548 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2704bcb-e1e3-4290-a2cc-596fa008b4ae None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:15.809584 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 437/872] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 2 19:01:15 2022] GET /identity/v3/role_assignments?user.id=04f64813f09348eb91cb7bb2869f7612&scope.project.id=8c04020eaebd4647804cf8653f32d9e5&role.id=0e56fba112f94ebba363ffab7f4aa9fe => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:15.811830 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aae930e-c6c2-4037-ac7e-132c8b3a86d3 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:15.843087 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:15.945616 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aae930e-c6c2-4037-ac7e-132c8b3a86d3 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:15.991775 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a156e2af-b4e8-43ed-90d6-5ca86e250390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:15.993491 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156e2af-b4e8-43ed-90d6-5ca86e250390 None admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:15.994023 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156e2af-b4e8-43ed-90d6-5ca86e250390 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:15.994560 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156e2af-b4e8-43ed-90d6-5ca86e250390 None admin] PATH_INFO: `/v3/services` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:15.995561 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a156e2af-b4e8-43ed-90d6-5ca86e250390 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:15.997435 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a156e2af-b4e8-43ed-90d6-5ca86e250390 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:16.040524 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4333dd5-8663-49b4-b343-2fb072d299a2 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:16.040884 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4333dd5-8663-49b4-b343-2fb072d299a2 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:16.041095 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4333dd5-8663-49b4-b343-2fb072d299a2 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:16.042840 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 438/873] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:01:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:01:16.049755 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeb0ec3-c5c8-45b3-a487-580dfd92d22e None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:16.050040 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeb0ec3-c5c8-45b3-a487-580dfd92d22e None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:16.052604 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeb0ec3-c5c8-45b3-a487-580dfd92d22e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:16.068807 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-5aeb0ec3-c5c8-45b3-a487-580dfd92d22e None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:01:16.070027 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-5aeb0ec3-c5c8-45b3-a487-580dfd92d22e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:01:16.085207 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 436/874] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Jun 2 19:01:15 2022] POST /identity/v3/services => generated 251 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 02 19:01:16.186313 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 439/875] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:01:16 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 02 19:01:16.211242 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a156e2af-b4e8-43ed-90d6-5ca86e250390 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:16.319007 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:16.342186 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a156e2af-b4e8-43ed-90d6-5ca86e250390 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:16.385341 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247095ad-5a82-406a-a8ca-a977b5580bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:16.386176 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247095ad-5a82-406a-a8ca-a977b5580bf4 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:16.386504 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247095ad-5a82-406a-a8ca-a977b5580bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:16.386816 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247095ad-5a82-406a-a8ca-a977b5580bf4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:16.387434 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247095ad-5a82-406a-a8ca-a977b5580bf4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:16.388858 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247095ad-5a82-406a-a8ca-a977b5580bf4 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:16.401825 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-247095ad-5a82-406a-a8ca-a977b5580bf4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 02 19:01:16.401825 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:16.401825 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 02 19:01:16.401825 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 02 19:01:16.401825 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 02 19:01:16.401825 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 02 19:01:16.401825 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 02 19:01:16.401825 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 02 19:01:16.401825 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 02 19:01:16.401825 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:01:16.401825 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 02 19:01:16.401825 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:01:16.401825 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:16.401825 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:16.401825 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:16.401825 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:16.401825 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:16.402906 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:16.402906 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:16.402906 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:16.402906 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:16.402906 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:16.402906 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:16.402906 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 02 19:01:16.402906 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 02 19:01:16.402906 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 02 19:01:16.402906 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 02 19:01:16.402906 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:16.402906 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:16.402906 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:16.402906 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:16.402906 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:16.402906 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:16.403950 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:16.403950 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:16.403950 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:16.403950 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:16.403950 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:16.403950 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:16.403950 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:16.403950 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:16.403950 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:16.403950 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:16.403950 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 02 19:01:16.403950 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 02 19:01:16.403950 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 02 19:01:16.403950 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:01:16.403950 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 437/876] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Jun 2 19:01:16 2022] GET /identity/v3/services/object-store => generated 93 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:16.409997 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeb0ec3-c5c8-45b3-a487-580dfd92d22e None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:16.442010 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:16.521392 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeb0ec3-c5c8-45b3-a487-580dfd92d22e None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:16.559660 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c757aa04-c141-42e5-a458-17ea4d518774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:16.560348 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c757aa04-c141-42e5-a458-17ea4d518774 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:16.560633 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c757aa04-c141-42e5-a458-17ea4d518774 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:16.560912 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c757aa04-c141-42e5-a458-17ea4d518774 None admin] PATH_INFO: `/v3/services` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:16.561643 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c757aa04-c141-42e5-a458-17ea4d518774 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:16.561957 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c757aa04-c141-42e5-a458-17ea4d518774 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:16.563410 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c757aa04-c141-42e5-a458-17ea4d518774 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:16.569043 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 440/877] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Jun 2 19:01:16 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:16.574166 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c757aa04-c141-42e5-a458-17ea4d518774 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:16.574668 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e15a434-30d1-4ed0-bc2f-787939aca4aa None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:16.575487 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e15a434-30d1-4ed0-bc2f-787939aca4aa None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:16.575894 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e15a434-30d1-4ed0-bc2f-787939aca4aa None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:16.577705 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 438/878] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:01:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:01:16.583616 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dbc621-864a-4e06-a108-1db39d62cf84 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:16.583857 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dbc621-864a-4e06-a108-1db39d62cf84 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:16.584291 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dbc621-864a-4e06-a108-1db39d62cf84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:16.604133 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-41dbc621-864a-4e06-a108-1db39d62cf84 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:01:16.605569 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-41dbc621-864a-4e06-a108-1db39d62cf84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:01:16.690904 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 439/879] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:01:16 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 02 19:01:16.705719 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:16.713702 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dbc621-864a-4e06-a108-1db39d62cf84 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:16.730650 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c757aa04-c141-42e5-a458-17ea4d518774 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:16.780831 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851ce455-531a-4dec-a801-c9fd48b6536e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:16.782254 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ce455-531a-4dec-a801-c9fd48b6536e None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:16.782830 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ce455-531a-4dec-a801-c9fd48b6536e None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:16.783579 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ce455-531a-4dec-a801-c9fd48b6536e None admin] PATH_INFO: `/v3/services` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:16.784923 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851ce455-531a-4dec-a801-c9fd48b6536e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:16.785823 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851ce455-531a-4dec-a801-c9fd48b6536e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:16.789637 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851ce455-531a-4dec-a801-c9fd48b6536e None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:16.803161 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 441/880] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Jun 2 19:01:16 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:01:16.815739 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:16.852037 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dbc621-864a-4e06-a108-1db39d62cf84 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:16.899870 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d25a23f-0bb7-4564-8820-5d7f76192baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:16.900575 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d25a23f-0bb7-4564-8820-5d7f76192baf None admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:16.900863 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d25a23f-0bb7-4564-8820-5d7f76192baf None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:16.901142 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d25a23f-0bb7-4564-8820-5d7f76192baf None admin] PATH_INFO: `/v3/services` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:16.901914 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d25a23f-0bb7-4564-8820-5d7f76192baf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:16.903392 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d25a23f-0bb7-4564-8820-5d7f76192baf None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:17.006291 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 440/881] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Jun 2 19:01:16 2022] POST /identity/v3/services => generated 269 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 02 19:01:18.452678 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b3f924-10f0-4e1d-8f2c-d4a78690563f None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:18.453195 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b3f924-10f0-4e1d-8f2c-d4a78690563f None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:18.453195 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b3f924-10f0-4e1d-8f2c-d4a78690563f None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:18.454677 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 442/882] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:01:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:01:18.460458 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf529a58-9c38-4929-a104-4d3c7a95b19a None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:18.460972 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf529a58-9c38-4929-a104-4d3c7a95b19a None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:18.461059 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf529a58-9c38-4929-a104-4d3c7a95b19a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:18.479521 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-bf529a58-9c38-4929-a104-4d3c7a95b19a None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:01:18.481124 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-bf529a58-9c38-4929-a104-4d3c7a95b19a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:01:18.551278 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0cd60-12ef-41f6-90aa-b61f8b0d9699 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:18.551968 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0cd60-12ef-41f6-90aa-b61f8b0d9699 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:18.552455 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0cd60-12ef-41f6-90aa-b61f8b0d9699 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:18.556188 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 443/883] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:01:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:01:18.560836 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df78e73a-d20e-43f1-8a2e-618ac9c072cf None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:18.561383 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df78e73a-d20e-43f1-8a2e-618ac9c072cf None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:18.561902 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df78e73a-d20e-43f1-8a2e-618ac9c072cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:18.564716 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 441/884] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:01:18 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 02 19:01:18.583243 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-df78e73a-d20e-43f1-8a2e-618ac9c072cf None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:01:18.584754 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-df78e73a-d20e-43f1-8a2e-618ac9c072cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:01:18.588002 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf529a58-9c38-4929-a104-4d3c7a95b19a None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:18.658157 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:18.666886 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 444/885] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:01:18 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 02 19:01:18.685243 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df78e73a-d20e-43f1-8a2e-618ac9c072cf None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:18.702796 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf529a58-9c38-4929-a104-4d3c7a95b19a None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:18.742239 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6405211-aef1-4e84-bf98-0dd1d5b30c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:18.743914 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6405211-aef1-4e84-bf98-0dd1d5b30c8c None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:18.744388 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6405211-aef1-4e84-bf98-0dd1d5b30c8c None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:18.744820 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6405211-aef1-4e84-bf98-0dd1d5b30c8c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:18.759581 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6405211-aef1-4e84-bf98-0dd1d5b30c8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:01:18.769118 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-b6405211-aef1-4e84-bf98-0dd1d5b30c8c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 02 19:01:18.769118 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:18.769118 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:18.769118 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:18.769118 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:18.769118 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:18.769118 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:18.769118 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:18.769118 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:18.769118 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:18.769118 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:18.769118 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:18.769118 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:01:18.769118 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:01:18.769118 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:01:18.769118 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:01:18.770321 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:01:18.770321 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:01:18.770321 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:01:18.770321 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:01:18.770321 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:18.770321 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:18.770321 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:18.770321 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:18.770321 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:18.770321 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:18.770321 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:18.770321 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:18.770321 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:18.770321 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:18.770321 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:18.770321 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:18.771381 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:18.771381 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:18.771381 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:18.771381 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:18.771381 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:01:18.771381 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:01:18.771381 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:01:18.771381 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:01:18.771381 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:01:18.771381 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:01:18.771381 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 02 19:01:18.771381 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:01:18.774676 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 442/886] 10.222.0.33 () {56 vars in 1120 bytes} [Thu Jun 2 19:01:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 02 19:01:18.780648 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6405211-aef1-4e84-bf98-0dd1d5b30c8c None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:18.786184 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:18.799808 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df78e73a-d20e-43f1-8a2e-618ac9c072cf None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:18.838046 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0e0874-befd-450a-a444-62114dd676b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:18.839068 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0e0874-befd-450a-a444-62114dd676b6 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:18.839618 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0e0874-befd-450a-a444-62114dd676b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:18.840114 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0e0874-befd-450a-a444-62114dd676b6 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:18.841035 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0e0874-befd-450a-a444-62114dd676b6 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:18.842945 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0e0874-befd-450a-a444-62114dd676b6 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:18.858418 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-ac0e0874-befd-450a-a444-62114dd676b6 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 02 19:01:18.858418 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:18.858418 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 02 19:01:18.858418 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 02 19:01:18.858418 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 02 19:01:18.858418 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 02 19:01:18.858418 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 02 19:01:18.858418 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 02 19:01:18.858418 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 02 19:01:18.858418 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:18.858418 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 02 19:01:18.858418 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:18.858418 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:18.858418 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:18.858418 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:18.858418 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:18.858418 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:18.859500 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:18.859500 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:18.859500 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:18.859500 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:18.859500 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:18.859500 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:18.859500 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 02 19:01:18.859500 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 02 19:01:18.859500 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 02 19:01:18.859500 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 02 19:01:18.859500 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:18.859500 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:18.859500 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:18.859500 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:18.859500 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:18.859500 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:18.860327 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:18.860327 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:18.860327 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:18.860327 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:18.860327 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:18.860327 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:18.860327 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:18.860327 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:18.860327 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:18.860327 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:18.860327 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 02 19:01:18.860327 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 02 19:01:18.860327 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 02 19:01:18.860327 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:18.868394 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 445/887] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 2 19:01:18 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:18.870795 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0e0874-befd-450a-a444-62114dd676b6 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:18.912933 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6405211-aef1-4e84-bf98-0dd1d5b30c8c None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:18.939828 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:18.959776 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c675952-b3c6-4f14-a387-8ea06327d78a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:18.959776 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c675952-b3c6-4f14-a387-8ea06327d78a None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:18.959776 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c675952-b3c6-4f14-a387-8ea06327d78a None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:18.959776 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c675952-b3c6-4f14-a387-8ea06327d78a None admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:18.960244 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c675952-b3c6-4f14-a387-8ea06327d78a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:18.960244 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c675952-b3c6-4f14-a387-8ea06327d78a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:18.960244 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c675952-b3c6-4f14-a387-8ea06327d78a None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:18.973034 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 443/888] 10.222.0.33 () {56 vars in 1115 bytes} [Thu Jun 2 19:01:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:18.974322 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c675952-b3c6-4f14-a387-8ea06327d78a None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:18.982333 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0e0874-befd-450a-a444-62114dd676b6 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:19.020190 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc19eba-f402-48df-80f5-ab6b85bd5643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:19.021124 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc19eba-f402-48df-80f5-ab6b85bd5643 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:19.021556 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc19eba-f402-48df-80f5-ab6b85bd5643 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:19.022003 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc19eba-f402-48df-80f5-ab6b85bd5643 None admin] PATH_INFO: `/v3/services` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:19.022911 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc19eba-f402-48df-80f5-ab6b85bd5643 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:19.023411 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc19eba-f402-48df-80f5-ab6b85bd5643 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:19.025041 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc19eba-f402-48df-80f5-ab6b85bd5643 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:19.035375 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 446/889] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 2 19:01:18 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:19.037026 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc19eba-f402-48df-80f5-ab6b85bd5643 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:19.054338 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:19.096514 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c675952-b3c6-4f14-a387-8ea06327d78a None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:19.152700 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b81536b-d9e1-4926-92c2-d50b5aa1aeda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:19.154394 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b81536b-d9e1-4926-92c2-d50b5aa1aeda None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:19.155009 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b81536b-d9e1-4926-92c2-d50b5aa1aeda None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:19.155437 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b81536b-d9e1-4926-92c2-d50b5aa1aeda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:19.171049 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b81536b-d9e1-4926-92c2-d50b5aa1aeda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:19.173632 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b81536b-d9e1-4926-92c2-d50b5aa1aeda None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:19.175497 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc19eba-f402-48df-80f5-ab6b85bd5643 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:19.197724 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-6b81536b-d9e1-4926-92c2-d50b5aa1aeda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:01:19.197724 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:19.197724 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:19.197724 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:19.197724 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:19.197724 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:19.197724 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:19.197724 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:19.197724 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:19.197724 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:19.197724 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:19.197724 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:19.197724 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:01:19.197724 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:01:19.197724 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:01:19.198657 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:01:19.198657 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:19.198657 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:19.198657 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:19.198657 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:19.198657 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:19.198657 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:19.198657 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:19.198657 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:19.198657 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:19.198657 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:19.198657 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:19.198657 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:19.198657 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:19.198657 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:19.198657 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:19.198657 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:19.199868 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:01:19.199868 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:01:19.199868 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:01:19.199868 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:01:19.199868 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 444/890] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:01:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:19.201137 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af27366a-a9f5-440a-89de-99bb73060c59 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:19.201428 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af27366a-a9f5-440a-89de-99bb73060c59 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:19.201711 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af27366a-a9f5-440a-89de-99bb73060c59 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:19.203696 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 445/891] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:01:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:01:19.205204 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af27366a-a9f5-440a-89de-99bb73060c59 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:19.217023 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d09646c-b5cb-4c5a-953b-0ebfa60cfdbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:19.217627 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d09646c-b5cb-4c5a-953b-0ebfa60cfdbc None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:19.217913 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d09646c-b5cb-4c5a-953b-0ebfa60cfdbc None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:19.218386 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d09646c-b5cb-4c5a-953b-0ebfa60cfdbc None admin] PATH_INFO: `/v3/services` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:19.219144 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d09646c-b5cb-4c5a-953b-0ebfa60cfdbc None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:19.219462 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d09646c-b5cb-4c5a-953b-0ebfa60cfdbc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:19.220862 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d09646c-b5cb-4c5a-953b-0ebfa60cfdbc None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:19.226415 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 447/892] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 2 19:01:19 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:19.228108 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95710bab-38d6-4ec8-ab48-2a2e2a0ff981 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:19.228403 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95710bab-38d6-4ec8-ab48-2a2e2a0ff981 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:19.228686 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95710bab-38d6-4ec8-ab48-2a2e2a0ff981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:19.247049 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-95710bab-38d6-4ec8-ab48-2a2e2a0ff981 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:01:19.248265 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-95710bab-38d6-4ec8-ab48-2a2e2a0ff981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:01:19.255852 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:19.329708 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af27366a-a9f5-440a-89de-99bb73060c59 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:19.338770 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 448/893] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:01:19 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 02 19:01:19.339954 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95710bab-38d6-4ec8-ab48-2a2e2a0ff981 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:19.354540 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:19.393683 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11212488-c8dd-4129-bd1d-e71cddef85d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:19.394366 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11212488-c8dd-4129-bd1d-e71cddef85d9 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:19.394652 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11212488-c8dd-4129-bd1d-e71cddef85d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:19.394981 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11212488-c8dd-4129-bd1d-e71cddef85d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:19.395699 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11212488-c8dd-4129-bd1d-e71cddef85d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:19.396000 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11212488-c8dd-4129-bd1d-e71cddef85d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:19.397402 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11212488-c8dd-4129-bd1d-e71cddef85d9 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:19.411817 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 446/894] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:01:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:19.412532 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11212488-c8dd-4129-bd1d-e71cddef85d9 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:19.463131 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95710bab-38d6-4ec8-ab48-2a2e2a0ff981 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:19.499481 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c49acb-65d8-4e87-9e9e-5a70e2682ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:19.500152 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c49acb-65d8-4e87-9e9e-5a70e2682ff0 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:19.500437 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c49acb-65d8-4e87-9e9e-5a70e2682ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:19.500712 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c49acb-65d8-4e87-9e9e-5a70e2682ff0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:19.501531 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c49acb-65d8-4e87-9e9e-5a70e2682ff0 None admin] RBAC: Adding query filter params (interface=public, service_id=22f83afbcaad4e3b93ea191ff02e8882, region_id=RegionOne) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:19.501841 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c49acb-65d8-4e87-9e9e-5a70e2682ff0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:19.503271 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c49acb-65d8-4e87-9e9e-5a70e2682ff0 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:19.511379 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 449/895] 10.222.0.33 () {56 vars in 1251 bytes} [Thu Jun 2 19:01:19 2022] GET /identity/v3/endpoints?service_id=22f83afbcaad4e3b93ea191ff02e8882&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:19.512205 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c49acb-65d8-4e87-9e9e-5a70e2682ff0 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:19.519255 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:19.526176 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11212488-c8dd-4129-bd1d-e71cddef85d9 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:19.575132 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3579a68e-d6a4-40ad-bec2-9dfc5c58f9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:19.575805 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3579a68e-d6a4-40ad-bec2-9dfc5c58f9fa None admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:19.576090 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3579a68e-d6a4-40ad-bec2-9dfc5c58f9fa None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:19.576365 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3579a68e-d6a4-40ad-bec2-9dfc5c58f9fa None admin] PATH_INFO: `/v3/services` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:19.577096 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3579a68e-d6a4-40ad-bec2-9dfc5c58f9fa None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:19.578508 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3579a68e-d6a4-40ad-bec2-9dfc5c58f9fa None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:19.598342 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:19.629469 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c49acb-65d8-4e87-9e9e-5a70e2682ff0 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:19.634201 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 447/896] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Jun 2 19:01:19 2022] POST /identity/v3/services => generated 241 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:01:19.635594 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6724afbd-568d-4fea-862c-06f25dcb03c9 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:19.635888 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6724afbd-568d-4fea-862c-06f25dcb03c9 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:19.636170 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6724afbd-568d-4fea-862c-06f25dcb03c9 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:19.637620 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 448/897] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:01:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:01:19.638199 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6724afbd-568d-4fea-862c-06f25dcb03c9 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:19.684642 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0959f82-7f18-4ab6-a093-581f6984efe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:19.686295 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0959f82-7f18-4ab6-a093-581f6984efe4 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:19.686987 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0959f82-7f18-4ab6-a093-581f6984efe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:19.687604 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0959f82-7f18-4ab6-a093-581f6984efe4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:19.701813 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0959f82-7f18-4ab6-a093-581f6984efe4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:19.704584 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0959f82-7f18-4ab6-a093-581f6984efe4 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:19.722271 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-a0959f82-7f18-4ab6-a093-581f6984efe4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 02 19:01:19.722271 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:19.722271 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:19.722271 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:19.722271 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:19.722271 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:19.722271 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:19.722271 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:19.722271 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:19.722271 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:19.722271 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:19.722271 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:19.722271 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 02 19:01:19.722271 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 02 19:01:19.722271 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 02 19:01:19.722271 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 02 19:01:19.724008 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:19.724008 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:19.724008 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 02 19:01:19.724008 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:01:19.724008 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 02 19:01:19.724008 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:01:19.724008 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:19.724008 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:19.724008 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:19.724008 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:19.724008 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:19.724008 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:19.724008 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:19.724008 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:19.724008 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:19.724008 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:19.724008 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:19.724008 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:19.725787 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:19.725787 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:19.725787 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 02 19:01:19.725787 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 02 19:01:19.725787 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 02 19:01:19.725787 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 02 19:01:19.725787 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 02 19:01:19.725787 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 02 19:01:19.725787 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 02 19:01:19.725787 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:19.725787 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 450/898] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Jun 2 19:01:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:19.725787 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcb82b3-8032-4035-9882-ff7f34d8fe38 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:19.725787 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcb82b3-8032-4035-9882-ff7f34d8fe38 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:19.729031 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcb82b3-8032-4035-9882-ff7f34d8fe38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:19.747931 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-8dcb82b3-8032-4035-9882-ff7f34d8fe38 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:01:19.749228 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-8dcb82b3-8032-4035-9882-ff7f34d8fe38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:01:19.787361 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6724afbd-568d-4fea-862c-06f25dcb03c9 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:19.837734 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959818ec-3664-4617-9550-06630bf2b70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:19.838846 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959818ec-3664-4617-9550-06630bf2b70c None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:19.839306 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959818ec-3664-4617-9550-06630bf2b70c None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:19.839743 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959818ec-3664-4617-9550-06630bf2b70c None admin] PATH_INFO: `/v3/services` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:19.841200 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959818ec-3664-4617-9550-06630bf2b70c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:19.843003 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959818ec-3664-4617-9550-06630bf2b70c None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:19.854140 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 451/899] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:01:19 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 02 19:01:19.855622 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcb82b3-8032-4035-9882-ff7f34d8fe38 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:19.855758 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 449/900] 10.222.0.33 () {56 vars in 1087 bytes} [Thu Jun 2 19:01:19 2022] GET /identity/v3/services => generated 1497 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:01:19.897582 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959818ec-3664-4617-9550-06630bf2b70c None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:19.930350 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:19.948621 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:20.016231 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcb82b3-8032-4035-9882-ff7f34d8fe38 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:20.045947 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959818ec-3664-4617-9550-06630bf2b70c None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:20.066482 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53ed9c8-fc0d-4af8-bf4c-07c779e147a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:20.067507 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ed9c8-fc0d-4af8-bf4c-07c779e147a3 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:20.067803 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ed9c8-fc0d-4af8-bf4c-07c779e147a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:20.068088 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ed9c8-fc0d-4af8-bf4c-07c779e147a3 None admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:20.069493 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53ed9c8-fc0d-4af8-bf4c-07c779e147a3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:20.069818 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53ed9c8-fc0d-4af8-bf4c-07c779e147a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:20.071926 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53ed9c8-fc0d-4af8-bf4c-07c779e147a3 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:20.088780 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab4ebcb-8859-4a14-9e3d-7e4d32942cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:20.089669 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab4ebcb-8859-4a14-9e3d-7e4d32942cf2 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:20.089953 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab4ebcb-8859-4a14-9e3d-7e4d32942cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:20.090226 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab4ebcb-8859-4a14-9e3d-7e4d32942cf2 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:20.090929 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab4ebcb-8859-4a14-9e3d-7e4d32942cf2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:20.092409 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab4ebcb-8859-4a14-9e3d-7e4d32942cf2 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:20.097057 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 452/901] 10.222.0.33 () {56 vars in 1149 bytes} [Thu Jun 2 19:01:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:20.103274 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53ed9c8-fc0d-4af8-bf4c-07c779e147a3 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:20.105678 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-6ab4ebcb-8859-4a14-9e3d-7e4d32942cf2 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 02 19:01:20.105678 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:20.105678 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 02 19:01:20.105678 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 02 19:01:20.105678 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 02 19:01:20.105678 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 02 19:01:20.105678 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 02 19:01:20.105678 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 02 19:01:20.105678 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 02 19:01:20.105678 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:01:20.105678 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 02 19:01:20.105678 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:01:20.105678 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:20.105678 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:20.105678 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:20.105678 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:20.107412 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:20.107412 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:20.107412 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:20.107412 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:20.107412 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:20.107412 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:20.107412 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:20.107412 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 02 19:01:20.107412 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 02 19:01:20.107412 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 02 19:01:20.107412 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 02 19:01:20.107412 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:20.107412 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:20.107412 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:20.107412 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:20.107412 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:20.108948 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:20.108948 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:20.108948 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:20.108948 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:20.108948 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:20.108948 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:20.108948 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:20.108948 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:20.108948 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:20.108948 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:20.108948 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:20.108948 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 02 19:01:20.108948 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 02 19:01:20.108948 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 02 19:01:20.108948 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:01:20.108948 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 450/902] 10.222.0.33 () {56 vars in 1132 bytes} [Thu Jun 2 19:01:19 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:20.123313 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab4ebcb-8859-4a14-9e3d-7e4d32942cf2 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:20.259010 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53ed9c8-fc0d-4af8-bf4c-07c779e147a3 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:20.285176 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:20.301715 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:20.314367 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab4ebcb-8859-4a14-9e3d-7e4d32942cf2 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:20.345121 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fa4335-73ee-4794-8a80-15a0a5e8e5f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:20.346813 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa4335-73ee-4794-8a80-15a0a5e8e5f4 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:20.348388 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa4335-73ee-4794-8a80-15a0a5e8e5f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:20.348695 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa4335-73ee-4794-8a80-15a0a5e8e5f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:20.378523 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe2782-7f19-46cc-9ada-aa69358fb930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:20.379801 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe2782-7f19-46cc-9ada-aa69358fb930 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:20.380626 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe2782-7f19-46cc-9ada-aa69358fb930 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:20.381168 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe2782-7f19-46cc-9ada-aa69358fb930 None admin] PATH_INFO: `/v3/services` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:20.382504 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fe2782-7f19-46cc-9ada-aa69358fb930 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:20.383130 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fe2782-7f19-46cc-9ada-aa69358fb930 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:20.386311 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fe2782-7f19-46cc-9ada-aa69358fb930 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:20.403198 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 451/903] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Jun 2 19:01:20 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:20.404836 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe2782-7f19-46cc-9ada-aa69358fb930 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:20.494524 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fa4335-73ee-4794-8a80-15a0a5e8e5f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:20.498733 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fa4335-73ee-4794-8a80-15a0a5e8e5f4 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:20.584139 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe2782-7f19-46cc-9ada-aa69358fb930 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:20.652141 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e38071-1a2a-4e8b-99a5-e4b4e7c6da67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:20.653338 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e38071-1a2a-4e8b-99a5-e4b4e7c6da67 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:20.653884 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e38071-1a2a-4e8b-99a5-e4b4e7c6da67 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:20.654428 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e38071-1a2a-4e8b-99a5-e4b4e7c6da67 None admin] PATH_INFO: `/v3/services` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:20.656698 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e38071-1a2a-4e8b-99a5-e4b4e7c6da67 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:20.657315 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e38071-1a2a-4e8b-99a5-e4b4e7c6da67 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:20.659876 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e38071-1a2a-4e8b-99a5-e4b4e7c6da67 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:20.674524 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 452/904] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Jun 2 19:01:20 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:20.677987 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e38071-1a2a-4e8b-99a5-e4b4e7c6da67 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:20.710243 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:20.716458 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 453/905] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 2 19:01:20 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 614 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 02 19:01:20.729090 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fa4335-73ee-4794-8a80-15a0a5e8e5f4 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:20.745761 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:20.865686 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fa4335-73ee-4794-8a80-15a0a5e8e5f4 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:20.899539 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e38071-1a2a-4e8b-99a5-e4b4e7c6da67 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:20.940892 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df22e918-46fb-4a66-8c18-30d0358d162e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:20.942350 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df22e918-46fb-4a66-8c18-30d0358d162e None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:20.942986 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df22e918-46fb-4a66-8c18-30d0358d162e None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:20.943457 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673ca2e1-16cc-4f58-8d63-c40132ca0a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:20.943647 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df22e918-46fb-4a66-8c18-30d0358d162e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:20.944299 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673ca2e1-16cc-4f58-8d63-c40132ca0a82 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:20.944734 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673ca2e1-16cc-4f58-8d63-c40132ca0a82 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:20.945065 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673ca2e1-16cc-4f58-8d63-c40132ca0a82 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:20.946584 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673ca2e1-16cc-4f58-8d63-c40132ca0a82 None admin] RBAC: Adding query filter params (interface=public, service_id=45b1f68af87f4e64bddb047d12622378, region_id=RegionOne) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:20.947036 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673ca2e1-16cc-4f58-8d63-c40132ca0a82 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:20.949311 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673ca2e1-16cc-4f58-8d63-c40132ca0a82 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:20.962779 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 453/906] 10.222.0.33 () {56 vars in 1251 bytes} [Thu Jun 2 19:01:20 2022] GET /identity/v3/endpoints?service_id=45b1f68af87f4e64bddb047d12622378&interface=public®ion_id=RegionOne => generated 194 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:01:20.966403 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673ca2e1-16cc-4f58-8d63-c40132ca0a82 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:20.970909 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df22e918-46fb-4a66-8c18-30d0358d162e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:20.974198 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df22e918-46fb-4a66-8c18-30d0358d162e None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:21.012703 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-df22e918-46fb-4a66-8c18-30d0358d162e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:01:21.012703 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:21.012703 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:21.012703 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:21.012703 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:21.012703 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:21.012703 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:21.012703 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:21.012703 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:21.012703 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:21.012703 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:21.012703 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:21.012703 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:01:21.012703 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:01:21.012703 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:01:21.014416 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:01:21.014416 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:21.014416 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:21.014416 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:21.014416 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:21.014416 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:21.014416 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:21.014416 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:21.014416 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:21.014416 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:21.014416 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:21.014416 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:21.014416 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:21.014416 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:21.014416 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:21.014416 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:21.014416 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:21.016450 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:01:21.016450 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:01:21.016450 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:01:21.016450 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:21.020061 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 454/907] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:01:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:21.028617 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df22e918-46fb-4a66-8c18-30d0358d162e None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:21.062490 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:21.089930 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673ca2e1-16cc-4f58-8d63-c40132ca0a82 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:21.141172 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2b7d5e-059f-48f5-b263-3bca59f51728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:21.142100 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b7d5e-059f-48f5-b263-3bca59f51728 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:21.142613 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b7d5e-059f-48f5-b263-3bca59f51728 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:21.143065 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b7d5e-059f-48f5-b263-3bca59f51728 None admin] PATH_INFO: `/v3/services` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:21.143982 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2b7d5e-059f-48f5-b263-3bca59f51728 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:21.145646 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2b7d5e-059f-48f5-b263-3bca59f51728 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:21.158904 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 454/908] 10.222.0.33 () {56 vars in 1087 bytes} [Thu Jun 2 19:01:20 2022] GET /identity/v3/services => generated 1497 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:01:21.194210 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:21.267088 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df22e918-46fb-4a66-8c18-30d0358d162e None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:21.338102 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d259e-6666-4f56-999c-38692bdad43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:21.339301 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d259e-6666-4f56-999c-38692bdad43c None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:21.339799 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d259e-6666-4f56-999c-38692bdad43c None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:21.340290 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d259e-6666-4f56-999c-38692bdad43c None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:21.341631 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432d259e-6666-4f56-999c-38692bdad43c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:21.342163 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432d259e-6666-4f56-999c-38692bdad43c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:21.344912 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432d259e-6666-4f56-999c-38692bdad43c None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:21.368505 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 455/909] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:01:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:01:21.376176 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2b7d5e-059f-48f5-b263-3bca59f51728 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:21.502891 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2b7d5e-059f-48f5-b263-3bca59f51728 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:21.568787 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24827c35-e3d4-466f-b7f9-96295733de7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:21.570137 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24827c35-e3d4-466f-b7f9-96295733de7b None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:21.570648 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24827c35-e3d4-466f-b7f9-96295733de7b None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:21.571297 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24827c35-e3d4-466f-b7f9-96295733de7b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:21.584772 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24827c35-e3d4-466f-b7f9-96295733de7b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:21.587812 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24827c35-e3d4-466f-b7f9-96295733de7b None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:21.592856 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:21.617222 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-24827c35-e3d4-466f-b7f9-96295733de7b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:01:21.617222 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:21.617222 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:21.617222 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:21.617222 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:21.617222 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:21.617222 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:21.617222 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:21.617222 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:21.617222 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:21.617222 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:21.617222 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:21.617222 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 02 19:01:21.617222 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 02 19:01:21.617222 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 02 19:01:21.618785 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 02 19:01:21.618785 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:21.618785 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:21.618785 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:21.618785 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:21.618785 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:21.618785 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:21.618785 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:21.618785 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:21.618785 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:21.618785 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:21.618785 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:21.618785 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:21.618785 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:21.618785 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:21.618785 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:21.618785 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:21.620053 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 02 19:01:21.620053 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 02 19:01:21.620053 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 02 19:01:21.620053 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 02 19:01:21.620053 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 02 19:01:21.620053 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 02 19:01:21.620053 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:01:21.620053 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:01:21.620916 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 455/910] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 2 19:01:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:21.630254 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d259e-6666-4f56-999c-38692bdad43c None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:21.826346 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d259e-6666-4f56-999c-38692bdad43c None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:21.880648 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:21.895365 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd839300-8ed6-4bbb-a3d6-3669cdc87735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:21.896566 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd839300-8ed6-4bbb-a3d6-3669cdc87735 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:21.897055 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd839300-8ed6-4bbb-a3d6-3669cdc87735 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:21.897535 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd839300-8ed6-4bbb-a3d6-3669cdc87735 None admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:21.899709 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd839300-8ed6-4bbb-a3d6-3669cdc87735 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:21.900234 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd839300-8ed6-4bbb-a3d6-3669cdc87735 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:21.903030 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd839300-8ed6-4bbb-a3d6-3669cdc87735 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:21.929173 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 456/911] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 2 19:01:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:21.939147 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24827c35-e3d4-466f-b7f9-96295733de7b None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:22.129538 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24827c35-e3d4-466f-b7f9-96295733de7b None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:22.190781 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f50279-5270-4734-9ff3-131db71ba766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:22.191907 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f50279-5270-4734-9ff3-131db71ba766 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:22.192403 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f50279-5270-4734-9ff3-131db71ba766 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:22.192907 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f50279-5270-4734-9ff3-131db71ba766 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:22.194486 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f50279-5270-4734-9ff3-131db71ba766 None admin] RBAC: Adding query filter params (role.id=8d5938aa1af04179bc44ffc0e0f376d9, scope.project.id=8c04020eaebd4647804cf8653f32d9e5, user.id=04f64813f09348eb91cb7bb2869f7612) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:22.195070 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f50279-5270-4734-9ff3-131db71ba766 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:22.197852 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f50279-5270-4734-9ff3-131db71ba766 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:22.209511 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 456/912] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 2 19:01:21 2022] GET /identity/v3/role_assignments?user.id=04f64813f09348eb91cb7bb2869f7612&scope.project.id=8c04020eaebd4647804cf8653f32d9e5&role.id=8d5938aa1af04179bc44ffc0e0f376d9 => generated 259 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:01:22.783363 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d9be50-8e8f-4b25-a3c5-e5c264f4d61e None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:22.783984 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d9be50-8e8f-4b25-a3c5-e5c264f4d61e None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:22.784070 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d9be50-8e8f-4b25-a3c5-e5c264f4d61e None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:22.786178 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 457/913] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:01:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 02 19:01:22.793261 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d290125a-c668-42dd-93bc-8bbaa92a906e None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:22.793938 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d290125a-c668-42dd-93bc-8bbaa92a906e None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:22.793938 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d290125a-c668-42dd-93bc-8bbaa92a906e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:22.815568 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-d290125a-c668-42dd-93bc-8bbaa92a906e None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:01:22.816822 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-d290125a-c668-42dd-93bc-8bbaa92a906e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:01:22.911392 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:22.922533 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 457/914] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:01:22 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 02 19:01:22.946459 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d9be50-8e8f-4b25-a3c5-e5c264f4d61e None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:23.040659 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b5973-8f82-4526-8beb-31c0bf2766da None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:23.040962 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b5973-8f82-4526-8beb-31c0bf2766da None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:23.041149 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b5973-8f82-4526-8beb-31c0bf2766da None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:23.042923 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 458/915] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:01:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:01:23.051675 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f7e6ca-f390-4791-8f8e-acb198422ed1 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:23.052121 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f7e6ca-f390-4791-8f8e-acb198422ed1 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:23.053610 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f7e6ca-f390-4791-8f8e-acb198422ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:23.078572 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-e6f7e6ca-f390-4791-8f8e-acb198422ed1 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:01:23.080824 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-e6f7e6ca-f390-4791-8f8e-acb198422ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:01:23.091537 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d9be50-8e8f-4b25-a3c5-e5c264f4d61e None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:23.136271 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c192b974-0bdc-4d4d-a6c7-975257c94b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:23.138290 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c192b974-0bdc-4d4d-a6c7-975257c94b1d None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:23.138290 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c192b974-0bdc-4d4d-a6c7-975257c94b1d None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:23.138290 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c192b974-0bdc-4d4d-a6c7-975257c94b1d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:23.138901 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c192b974-0bdc-4d4d-a6c7-975257c94b1d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:23.140702 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c192b974-0bdc-4d4d-a6c7-975257c94b1d None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:23.155472 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-c192b974-0bdc-4d4d-a6c7-975257c94b1d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 02 19:01:23.155472 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:23.155472 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 02 19:01:23.155472 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 02 19:01:23.155472 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 02 19:01:23.155472 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 02 19:01:23.155472 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 02 19:01:23.155472 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 02 19:01:23.155472 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 02 19:01:23.155472 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:23.155472 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 02 19:01:23.155472 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:23.155472 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:23.155472 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:23.155472 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:23.155472 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:23.155472 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:23.157521 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:23.157521 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:23.157521 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:23.157521 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:23.157521 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:23.157521 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:23.157521 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 02 19:01:23.157521 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 02 19:01:23.157521 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 02 19:01:23.157521 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 02 19:01:23.157521 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:23.157521 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:23.157521 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:23.157521 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:23.157521 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:23.157521 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:23.159212 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:23.159212 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:23.159212 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:23.159212 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:23.159212 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:23.159212 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:23.159212 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:23.159212 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:23.159212 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:23.159212 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:23.159212 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 02 19:01:23.159212 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 02 19:01:23.159212 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 02 19:01:23.159212 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:23.159212 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 458/916] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 2 19:01:22 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:23.164144 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c192b974-0bdc-4d4d-a6c7-975257c94b1d None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:23.216800 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 459/917] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:01:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 02 19:01:23.230894 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:23.242407 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f7e6ca-f390-4791-8f8e-acb198422ed1 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:23.288647 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c192b974-0bdc-4d4d-a6c7-975257c94b1d None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:23.345002 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3ce42d-54cd-4faa-953f-c039650ccc44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:23.345636 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3ce42d-54cd-4faa-953f-c039650ccc44 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:23.346197 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3ce42d-54cd-4faa-953f-c039650ccc44 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:23.346741 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3ce42d-54cd-4faa-953f-c039650ccc44 None admin] PATH_INFO: `/v3/services` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:23.348152 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3ce42d-54cd-4faa-953f-c039650ccc44 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:23.348753 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3ce42d-54cd-4faa-953f-c039650ccc44 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:23.351492 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3ce42d-54cd-4faa-953f-c039650ccc44 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:23.361823 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 459/918] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 2 19:01:23 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:23.372535 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3ce42d-54cd-4faa-953f-c039650ccc44 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:23.458397 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f7e6ca-f390-4791-8f8e-acb198422ed1 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:23.509480 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:23.533546 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63532982-c420-495e-9c94-562a701c6c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:23.535061 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63532982-c420-495e-9c94-562a701c6c8c None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:23.535593 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63532982-c420-495e-9c94-562a701c6c8c None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:23.536107 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63532982-c420-495e-9c94-562a701c6c8c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:23.537330 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63532982-c420-495e-9c94-562a701c6c8c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:23.541068 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63532982-c420-495e-9c94-562a701c6c8c None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:23.570073 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-63532982-c420-495e-9c94-562a701c6c8c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 02 19:01:23.570073 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:23.570073 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 02 19:01:23.570073 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 02 19:01:23.570073 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 02 19:01:23.570073 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 02 19:01:23.570073 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 02 19:01:23.570073 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 02 19:01:23.570073 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 02 19:01:23.570073 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:01:23.570073 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 02 19:01:23.570073 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:01:23.570073 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:23.570073 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:23.570073 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:23.570073 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:23.570073 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:23.572588 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:23.572588 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:23.572588 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:23.572588 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:23.572588 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:23.572588 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:23.572588 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 02 19:01:23.572588 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 02 19:01:23.572588 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 02 19:01:23.572588 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 02 19:01:23.572588 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:23.572588 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:23.572588 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:23.572588 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:23.572588 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:23.572588 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:23.574180 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:23.574180 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:23.574180 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:23.574180 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:23.574180 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:23.574180 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:23.574180 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:23.574180 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:23.574180 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:23.574180 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:23.574180 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 02 19:01:23.574180 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 02 19:01:23.574180 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 02 19:01:23.574180 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:01:23.574180 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 460/919] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Jun 2 19:01:23 2022] GET /identity/v3/services/object-store => generated 93 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:23.580207 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63532982-c420-495e-9c94-562a701c6c8c None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:23.603182 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3ce42d-54cd-4faa-953f-c039650ccc44 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:23.643539 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:23.683674 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81183428-6343-4220-ae0a-acbc32a2f253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:23.685557 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81183428-6343-4220-ae0a-acbc32a2f253 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:23.686509 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81183428-6343-4220-ae0a-acbc32a2f253 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:23.687390 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81183428-6343-4220-ae0a-acbc32a2f253 None admin] PATH_INFO: `/v3/services` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:23.691015 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81183428-6343-4220-ae0a-acbc32a2f253 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:23.692102 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81183428-6343-4220-ae0a-acbc32a2f253 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:23.695342 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81183428-6343-4220-ae0a-acbc32a2f253 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:23.703475 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 460/920] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Jun 2 19:01:23 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:23.712404 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81183428-6343-4220-ae0a-acbc32a2f253 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:23.767395 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63532982-c420-495e-9c94-562a701c6c8c None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:23.812912 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662aebf4-60f1-4811-93c6-015e2ad22be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:23.814487 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662aebf4-60f1-4811-93c6-015e2ad22be4 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:23.815033 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662aebf4-60f1-4811-93c6-015e2ad22be4 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:23.815570 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662aebf4-60f1-4811-93c6-015e2ad22be4 None admin] PATH_INFO: `/v3/services` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:23.816497 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662aebf4-60f1-4811-93c6-015e2ad22be4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:23.817060 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662aebf4-60f1-4811-93c6-015e2ad22be4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:23.818676 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662aebf4-60f1-4811-93c6-015e2ad22be4 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:23.825417 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 461/921] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Jun 2 19:01:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:23.832085 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662aebf4-60f1-4811-93c6-015e2ad22be4 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:23.881562 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81183428-6343-4220-ae0a-acbc32a2f253 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:23.895820 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:23.921736 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f7b594-2337-41e4-bf31-ca50aac75d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:23.922979 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f7b594-2337-41e4-bf31-ca50aac75d9f None admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:23.923263 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f7b594-2337-41e4-bf31-ca50aac75d9f None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:23.923537 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f7b594-2337-41e4-bf31-ca50aac75d9f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:23.924823 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f7b594-2337-41e4-bf31-ca50aac75d9f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:23.926346 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f7b594-2337-41e4-bf31-ca50aac75d9f None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:23.949735 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662aebf4-60f1-4811-93c6-015e2ad22be4 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:23.986913 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca0894d-387d-4141-9186-8b5cfade13b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:23.987627 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca0894d-387d-4141-9186-8b5cfade13b2 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:23.987910 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca0894d-387d-4141-9186-8b5cfade13b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:23.988187 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca0894d-387d-4141-9186-8b5cfade13b2 None admin] PATH_INFO: `/v3/services` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:23.988903 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca0894d-387d-4141-9186-8b5cfade13b2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:23.989211 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca0894d-387d-4141-9186-8b5cfade13b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:23.990586 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca0894d-387d-4141-9186-8b5cfade13b2 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:24.000600 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 462/922] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Jun 2 19:01:23 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:24.002521 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca0894d-387d-4141-9186-8b5cfade13b2 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:24.051446 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 461/923] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Jun 2 19:01:23 2022] POST /identity/v3/endpoints => generated 346 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:01:24.111781 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ddf324-8388-437c-9570-7077fc083384 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:24.112117 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ddf324-8388-437c-9570-7077fc083384 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:24.112363 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ddf324-8388-437c-9570-7077fc083384 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:24.114188 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 462/924] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:01:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 02 19:01:24.120880 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dd3a71-fd05-461b-9f23-a933899cc3df None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:24.121194 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dd3a71-fd05-461b-9f23-a933899cc3df None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:24.121452 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dd3a71-fd05-461b-9f23-a933899cc3df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:24.141719 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-27dd3a71-fd05-461b-9f23-a933899cc3df None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:01:24.142936 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-27dd3a71-fd05-461b-9f23-a933899cc3df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:01:24.147795 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca0894d-387d-4141-9186-8b5cfade13b2 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:24.170802 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:24.198982 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:24.199782 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1d095b-0050-4418-9a59-186f3f1e84c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:24.200547 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d095b-0050-4418-9a59-186f3f1e84c4 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:24.200835 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d095b-0050-4418-9a59-186f3f1e84c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:24.201343 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d095b-0050-4418-9a59-186f3f1e84c4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:24.202006 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1d095b-0050-4418-9a59-186f3f1e84c4 None admin] RBAC: Adding query filter params (interface=public, service_id=a046ad7a170942ed8d685212a3d684b5, region_id=RegionOne) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:24.202360 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1d095b-0050-4418-9a59-186f3f1e84c4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:24.203830 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1d095b-0050-4418-9a59-186f3f1e84c4 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:24.211767 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 463/925] 10.222.0.33 () {56 vars in 1251 bytes} [Thu Jun 2 19:01:24 2022] GET /identity/v3/endpoints?service_id=a046ad7a170942ed8d685212a3d684b5&interface=public®ion_id=RegionOne => generated 194 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:01:24.221533 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1d095b-0050-4418-9a59-186f3f1e84c4 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:24.252217 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 463/926] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:01:24 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 02 19:01:24.278306 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dd3a71-fd05-461b-9f23-a933899cc3df None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:24.380928 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1d095b-0050-4418-9a59-186f3f1e84c4 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:24.421461 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4460a92f-2dc8-49b4-92e5-80db66584725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:24.422092 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4460a92f-2dc8-49b4-92e5-80db66584725 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:24.422385 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4460a92f-2dc8-49b4-92e5-80db66584725 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:24.422670 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4460a92f-2dc8-49b4-92e5-80db66584725 None admin] PATH_INFO: `/v3/services` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:24.423396 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4460a92f-2dc8-49b4-92e5-80db66584725 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:24.424044 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dd3a71-fd05-461b-9f23-a933899cc3df None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:24.424958 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4460a92f-2dc8-49b4-92e5-80db66584725 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:24.438919 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 464/927] 10.222.0.33 () {56 vars in 1087 bytes} [Thu Jun 2 19:01:24 2022] GET /identity/v3/services => generated 1497 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:01:24.466958 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fad1ec-6882-4bb9-8843-b1b6ae787bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:24.467824 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fad1ec-6882-4bb9-8843-b1b6ae787bea None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:24.468111 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fad1ec-6882-4bb9-8843-b1b6ae787bea None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:24.468391 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fad1ec-6882-4bb9-8843-b1b6ae787bea None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:24.469148 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fad1ec-6882-4bb9-8843-b1b6ae787bea None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:24.470684 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fad1ec-6882-4bb9-8843-b1b6ae787bea None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:24.484879 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-64fad1ec-6882-4bb9-8843-b1b6ae787bea None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 02 19:01:24.484879 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:24.484879 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 02 19:01:24.484879 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 02 19:01:24.484879 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 02 19:01:24.484879 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 02 19:01:24.484879 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 02 19:01:24.484879 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 02 19:01:24.484879 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 02 19:01:24.484879 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:24.484879 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 02 19:01:24.484879 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:24.484879 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:24.484879 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:24.484879 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:24.484879 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:24.485896 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:24.485896 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:24.485896 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:24.485896 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:24.485896 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:24.485896 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:24.485896 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:24.485896 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 02 19:01:24.485896 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 02 19:01:24.485896 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 02 19:01:24.485896 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 02 19:01:24.485896 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:24.485896 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:24.485896 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:24.485896 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:24.485896 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:24.486742 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:24.486742 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:24.486742 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:24.486742 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:24.486742 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:24.486742 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:24.486742 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:24.486742 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:24.486742 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:24.486742 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:24.486742 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:24.486742 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 02 19:01:24.486742 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 02 19:01:24.486742 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 02 19:01:24.486742 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:24.486742 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 464/928] 10.222.0.33 () {56 vars in 1132 bytes} [Thu Jun 2 19:01:24 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:24.492953 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4460a92f-2dc8-49b4-92e5-80db66584725 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:24.544778 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:24.625850 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4460a92f-2dc8-49b4-92e5-80db66584725 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:24.686220 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355c7e04-72f1-42ce-bd3c-12367ae7d8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:24.687703 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355c7e04-72f1-42ce-bd3c-12367ae7d8e2 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:24.688128 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355c7e04-72f1-42ce-bd3c-12367ae7d8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:24.688562 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355c7e04-72f1-42ce-bd3c-12367ae7d8e2 None admin] PATH_INFO: `/v3/services` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:24.689703 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355c7e04-72f1-42ce-bd3c-12367ae7d8e2 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:24.690066 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355c7e04-72f1-42ce-bd3c-12367ae7d8e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:24.691489 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355c7e04-72f1-42ce-bd3c-12367ae7d8e2 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:24.700202 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 465/929] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Jun 2 19:01:24 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:24.708632 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fad1ec-6882-4bb9-8843-b1b6ae787bea None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:24.748235 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:24.831040 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fad1ec-6882-4bb9-8843-b1b6ae787bea None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:24.870015 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051ba6d8-8516-4a20-9ffc-366d4a235fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:24.870692 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051ba6d8-8516-4a20-9ffc-366d4a235fae None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:24.871011 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051ba6d8-8516-4a20-9ffc-366d4a235fae None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:24.871289 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051ba6d8-8516-4a20-9ffc-366d4a235fae None admin] PATH_INFO: `/v3/services` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:24.872008 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051ba6d8-8516-4a20-9ffc-366d4a235fae None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:24.872320 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051ba6d8-8516-4a20-9ffc-366d4a235fae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:24.873738 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051ba6d8-8516-4a20-9ffc-366d4a235fae None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:24.879876 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 465/930] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Jun 2 19:01:24 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:24.886064 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355c7e04-72f1-42ce-bd3c-12367ae7d8e2 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:24.955818 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c8b01b-b058-4de1-ba9c-781448efc369 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:24.956617 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c8b01b-b058-4de1-ba9c-781448efc369 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:24.956617 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c8b01b-b058-4de1-ba9c-781448efc369 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:24.957939 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 466/931] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:01:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:01:24.963987 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c91af-cc1b-4914-92c0-4d7ccb4d76cc None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:24.964276 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c91af-cc1b-4914-92c0-4d7ccb4d76cc None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:24.964557 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c91af-cc1b-4914-92c0-4d7ccb4d76cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:24.983323 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-306c91af-cc1b-4914-92c0-4d7ccb4d76cc None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:01:24.984251 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-306c91af-cc1b-4914-92c0-4d7ccb4d76cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:01:25.068163 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355c7e04-72f1-42ce-bd3c-12367ae7d8e2 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:25.070280 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 467/932] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:01:24 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 02 19:01:25.072415 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:25.092823 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306c91af-cc1b-4914-92c0-4d7ccb4d76cc None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:25.124467 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be90587-904c-4d14-a06f-1a93cf9b967f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:25.125171 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be90587-904c-4d14-a06f-1a93cf9b967f None admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:25.125459 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be90587-904c-4d14-a06f-1a93cf9b967f None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:25.125737 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be90587-904c-4d14-a06f-1a93cf9b967f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:25.126504 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be90587-904c-4d14-a06f-1a93cf9b967f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:25.128034 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be90587-904c-4d14-a06f-1a93cf9b967f None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:25.137014 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:25.218276 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306c91af-cc1b-4914-92c0-4d7ccb4d76cc None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:25.238578 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 466/933] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Jun 2 19:01:24 2022] POST /identity/v3/endpoints => generated 347 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:01:25.273479 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb447faa-08ed-4745-b321-d7e7ab629562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:25.274427 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb447faa-08ed-4745-b321-d7e7ab629562 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:25.274716 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb447faa-08ed-4745-b321-d7e7ab629562 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:25.275065 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb447faa-08ed-4745-b321-d7e7ab629562 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:25.288477 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb447faa-08ed-4745-b321-d7e7ab629562 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:01:25.298035 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-eb447faa-08ed-4745-b321-d7e7ab629562 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 02 19:01:25.298035 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:25.298035 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:25.298035 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:25.298035 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:25.298035 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:25.298035 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:25.298035 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:25.298035 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:25.298035 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:25.298035 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:25.298035 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:25.298035 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:01:25.298035 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:01:25.298035 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:01:25.298035 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:01:25.299123 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:01:25.299123 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:01:25.299123 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:01:25.299123 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:01:25.299123 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:25.299123 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:25.299123 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:25.299123 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:25.299123 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:25.299123 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:25.299123 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:25.299123 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:25.299123 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:25.299123 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:25.299123 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:25.299123 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:25.300274 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:25.300274 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:25.300274 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:25.300274 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:25.300274 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:01:25.300274 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:01:25.300274 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:01:25.300274 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:01:25.300274 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:01:25.300274 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:01:25.300274 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 02 19:01:25.300274 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:25.300274 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 468/934] 10.222.0.33 () {56 vars in 1120 bytes} [Thu Jun 2 19:01:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 02 19:01:25.306881 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be90587-904c-4d14-a06f-1a93cf9b967f None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:25.431397 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be90587-904c-4d14-a06f-1a93cf9b967f None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:25.470289 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ce3f21-029c-44aa-876d-930ccb1b14a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:25.471423 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce3f21-029c-44aa-876d-930ccb1b14a8 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:25.471719 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce3f21-029c-44aa-876d-930ccb1b14a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:25.472011 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce3f21-029c-44aa-876d-930ccb1b14a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:25.472792 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ce3f21-029c-44aa-876d-930ccb1b14a8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:25.473106 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ce3f21-029c-44aa-876d-930ccb1b14a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:25.474503 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ce3f21-029c-44aa-876d-930ccb1b14a8 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:25.484703 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 467/935] 10.222.0.33 () {56 vars in 1115 bytes} [Thu Jun 2 19:01:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:25.490541 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb447faa-08ed-4745-b321-d7e7ab629562 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:25.585606 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:25.607885 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb447faa-08ed-4745-b321-d7e7ab629562 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:25.667147 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11661656-0905-4f23-bc21-488e81a7b4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:25.669024 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11661656-0905-4f23-bc21-488e81a7b4a2 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:25.669826 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11661656-0905-4f23-bc21-488e81a7b4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:25.670644 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11661656-0905-4f23-bc21-488e81a7b4a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:25.683617 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11661656-0905-4f23-bc21-488e81a7b4a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:25.685432 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11661656-0905-4f23-bc21-488e81a7b4a2 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:25.704623 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-11661656-0905-4f23-bc21-488e81a7b4a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:01:25.704623 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:25.704623 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:25.704623 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:25.704623 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:25.704623 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:25.704623 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:25.704623 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:25.704623 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:25.704623 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:25.704623 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:25.704623 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:25.704623 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:01:25.704623 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:01:25.704623 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:01:25.705647 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:01:25.705647 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:25.705647 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:25.705647 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:25.705647 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:25.705647 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:25.705647 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:25.705647 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:25.705647 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:25.705647 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:25.705647 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:25.705647 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:25.705647 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:25.705647 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:25.705647 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:25.705647 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:25.705647 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:25.706620 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:01:25.706620 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:01:25.706620 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:01:25.706620 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:25.709112 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 469/936] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:01:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:25.715137 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ce3f21-029c-44aa-876d-930ccb1b14a8 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:25.747520 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:25.839466 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ce3f21-029c-44aa-876d-930ccb1b14a8 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:25.882820 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd536a9b-1631-4f71-8bd9-325fdb618691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:25.884577 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd536a9b-1631-4f71-8bd9-325fdb618691 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:25.885082 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd536a9b-1631-4f71-8bd9-325fdb618691 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:25.885573 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd536a9b-1631-4f71-8bd9-325fdb618691 None admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:25.886691 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd536a9b-1631-4f71-8bd9-325fdb618691 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:25.887714 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd536a9b-1631-4f71-8bd9-325fdb618691 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:25.889968 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd536a9b-1631-4f71-8bd9-325fdb618691 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:25.906146 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 468/937] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:01:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:25.911865 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11661656-0905-4f23-bc21-488e81a7b4a2 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:26.076636 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11661656-0905-4f23-bc21-488e81a7b4a2 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:26.135198 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:26.143569 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8353c38-243b-4992-8eb3-d607e5f1403b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:26.144508 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8353c38-243b-4992-8eb3-d607e5f1403b None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:26.144854 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8353c38-243b-4992-8eb3-d607e5f1403b None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:26.145132 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8353c38-243b-4992-8eb3-d607e5f1403b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:26.154968 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8353c38-243b-4992-8eb3-d607e5f1403b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:26.156636 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8353c38-243b-4992-8eb3-d607e5f1403b None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:26.173892 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-d8353c38-243b-4992-8eb3-d607e5f1403b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 02 19:01:26.173892 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:26.173892 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:26.173892 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:26.173892 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:26.173892 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:26.173892 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:26.173892 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:26.173892 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:26.173892 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:26.173892 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:26.173892 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:26.173892 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 02 19:01:26.173892 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 02 19:01:26.173892 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 02 19:01:26.173892 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 02 19:01:26.174925 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:26.174925 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:26.174925 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 02 19:01:26.174925 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:01:26.174925 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 02 19:01:26.174925 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:01:26.174925 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:26.174925 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:26.174925 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:26.174925 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:26.174925 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:26.174925 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:26.174925 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:26.174925 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:26.174925 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:26.174925 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:26.174925 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:26.174925 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:26.176040 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:26.176040 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:26.176040 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 02 19:01:26.176040 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 02 19:01:26.176040 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 02 19:01:26.176040 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 02 19:01:26.176040 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 02 19:01:26.176040 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 02 19:01:26.176040 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 02 19:01:26.176040 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:26.176040 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 470/938] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Jun 2 19:01:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:26.187922 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd536a9b-1631-4f71-8bd9-325fdb618691 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:26.280896 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:26.302928 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd536a9b-1631-4f71-8bd9-325fdb618691 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:26.341099 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c73e08-0d44-4642-89ee-81efcc9b63dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:26.341706 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c73e08-0d44-4642-89ee-81efcc9b63dc None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:26.341988 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c73e08-0d44-4642-89ee-81efcc9b63dc None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:26.342262 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c73e08-0d44-4642-89ee-81efcc9b63dc None admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:26.343180 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c73e08-0d44-4642-89ee-81efcc9b63dc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:26.343490 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c73e08-0d44-4642-89ee-81efcc9b63dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:26.345158 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c73e08-0d44-4642-89ee-81efcc9b63dc None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:26.365449 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 469/939] 10.222.0.33 () {56 vars in 1149 bytes} [Thu Jun 2 19:01:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:26.371273 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8353c38-243b-4992-8eb3-d607e5f1403b None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:26.500815 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8353c38-243b-4992-8eb3-d607e5f1403b None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:26.539688 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8de3b5-e511-418d-928d-465bc3099ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:26.541076 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8de3b5-e511-418d-928d-465bc3099ff2 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:26.541363 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8de3b5-e511-418d-928d-465bc3099ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:26.541641 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8de3b5-e511-418d-928d-465bc3099ff2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:26.554044 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8de3b5-e511-418d-928d-465bc3099ff2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:26.555664 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8de3b5-e511-418d-928d-465bc3099ff2 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:26.582096 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-fa8de3b5-e511-418d-928d-465bc3099ff2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:01:26.582096 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:26.582096 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:26.582096 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:26.582096 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:26.582096 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:26.582096 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:26.582096 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:26.582096 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:26.582096 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:26.582096 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:26.582096 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:26.582096 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:01:26.582096 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:01:26.582096 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:01:26.584275 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:01:26.584275 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:26.584275 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:26.584275 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:26.584275 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:26.584275 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:26.584275 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:26.584275 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:26.584275 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:26.584275 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:26.584275 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:26.584275 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:26.584275 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:26.584275 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:26.584275 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:26.584275 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:26.584275 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:26.587330 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:01:26.587330 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:01:26.587330 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:01:26.587330 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:26.587330 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 471/940] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:01:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:26.592626 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c73e08-0d44-4642-89ee-81efcc9b63dc None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:26.640947 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d247d588-102e-47d9-a1cc-93eaeb2c41f4 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:26.641240 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d247d588-102e-47d9-a1cc-93eaeb2c41f4 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:26.641676 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d247d588-102e-47d9-a1cc-93eaeb2c41f4 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:26.643633 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 472/941] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:01:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:01:26.664407 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38e67f0-0723-4ec0-b2e2-016ad32b3b1f None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:26.664907 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38e67f0-0723-4ec0-b2e2-016ad32b3b1f None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:26.665228 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38e67f0-0723-4ec0-b2e2-016ad32b3b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:26.700609 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-f38e67f0-0723-4ec0-b2e2-016ad32b3b1f None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:01:26.701981 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-f38e67f0-0723-4ec0-b2e2-016ad32b3b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:01:26.763753 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:26.766643 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c73e08-0d44-4642-89ee-81efcc9b63dc None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:26.807733 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 473/942] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:01:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 02 19:01:26.839494 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6af1d0-fbaa-4030-9ee8-03d0af264612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:26.839722 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38e67f0-0723-4ec0-b2e2-016ad32b3b1f None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:26.840715 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6af1d0-fbaa-4030-9ee8-03d0af264612 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:26.841283 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6af1d0-fbaa-4030-9ee8-03d0af264612 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:26.841763 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6af1d0-fbaa-4030-9ee8-03d0af264612 None admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:26.842827 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6af1d0-fbaa-4030-9ee8-03d0af264612 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:26.843353 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6af1d0-fbaa-4030-9ee8-03d0af264612 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:26.845012 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6af1d0-fbaa-4030-9ee8-03d0af264612 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:26.855942 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:26.861105 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 470/943] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:01:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:26.867234 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6af1d0-fbaa-4030-9ee8-03d0af264612 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:26.979891 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38e67f0-0723-4ec0-b2e2-016ad32b3b1f None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:26.990734 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6af1d0-fbaa-4030-9ee8-03d0af264612 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:27.020786 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e813c7-38b3-4de3-ba72-17a41945615b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:27.021704 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e813c7-38b3-4de3-ba72-17a41945615b None admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:27.021825 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e813c7-38b3-4de3-ba72-17a41945615b None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:27.022013 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e813c7-38b3-4de3-ba72-17a41945615b None admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:27.022721 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e813c7-38b3-4de3-ba72-17a41945615b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:27.024730 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e813c7-38b3-4de3-ba72-17a41945615b None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:27.026097 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: WARNING keystone.server.flask.common [None req-28e813c7-38b3-4de3-ba72-17a41945615b None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 02 19:01:27.033467 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a685e517-b8b7-4ec7-b923-daba74a5d1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:27.034272 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a685e517-b8b7-4ec7-b923-daba74a5d1dd None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:27.034556 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a685e517-b8b7-4ec7-b923-daba74a5d1dd None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:27.034875 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a685e517-b8b7-4ec7-b923-daba74a5d1dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:27.042316 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a685e517-b8b7-4ec7-b923-daba74a5d1dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:27.043905 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a685e517-b8b7-4ec7-b923-daba74a5d1dd None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:27.058142 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-a685e517-b8b7-4ec7-b923-daba74a5d1dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:01:27.058142 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:27.058142 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:27.058142 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:27.058142 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:27.058142 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:27.058142 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:27.058142 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:27.058142 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:27.058142 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:27.058142 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:27.058142 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:27.058142 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 02 19:01:27.058142 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 02 19:01:27.058142 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 02 19:01:27.063589 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 02 19:01:27.063589 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:27.063589 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:27.063589 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:27.063589 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:27.063589 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:27.063589 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:27.063589 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:27.063589 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:27.063589 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:27.063589 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:27.063589 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:27.063589 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:27.063589 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:27.063589 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:27.063589 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:27.063589 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:27.064461 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 02 19:01:27.064461 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 02 19:01:27.064461 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 02 19:01:27.064461 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 02 19:01:27.064461 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 02 19:01:27.064461 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 02 19:01:27.064461 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:01:27.064461 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:01:27.064461 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 471/944] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 2 19:01:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 02 19:01:27.064461 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012bf41-884b-4701-b81e-4b5ecff18b69 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:27.064461 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012bf41-884b-4701-b81e-4b5ecff18b69 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:27.065029 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012bf41-884b-4701-b81e-4b5ecff18b69 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:27.065029 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 472/945] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:01:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:01:27.065672 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c012bf41-884b-4701-b81e-4b5ecff18b69 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:27.089307 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.sql.core [None req-28e813c7-38b3-4de3-ba72-17a41945615b None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 02 19:01:27.089307 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 02 19:01:27.089307 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [parameters: {'id': 'b2138ffa8fc04514b25cb811c8c46c6e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 02 19:01:27.089307 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80491) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 02 19:01:27.097140 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-28e813c7-38b3-4de3-ba72-17a41945615b None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b2138ffa8fc04514b25cb811c8c46c6e.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b2138ffa8fc04514b25cb811c8c46c6e. Jun 02 19:01:27.097140 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:27.097140 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 02 19:01:27.097140 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 02 19:01:27.097140 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 02 19:01:27.097140 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 02 19:01:27.097140 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 02 19:01:27.097140 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application result = self._query(query) Jun 02 19:01:27.097140 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 02 19:01:27.097140 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application conn.query(q) Jun 02 19:01:27.097140 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 02 19:01:27.097140 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 02 19:01:27.097140 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 02 19:01:27.098165 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application result.read() Jun 02 19:01:27.098165 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 02 19:01:27.098165 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 02 19:01:27.098165 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 02 19:01:27.098165 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 02 19:01:27.098165 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 02 19:01:27.098165 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 02 19:01:27.098165 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 02 19:01:27.098165 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 02 19:01:27.098165 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 02 19:01:27.098165 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:27.098165 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 02 19:01:27.098165 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:27.098165 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:27.098165 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 02 19:01:27.098165 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 02 19:01:27.098165 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 02 19:01:27.098165 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 02 19:01:27.099324 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 02 19:01:27.099324 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application next(self.gen) Jun 02 19:01:27.099324 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 02 19:01:27.099324 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application yield resource Jun 02 19:01:27.099324 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 02 19:01:27.099324 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application next(self.gen) Jun 02 19:01:27.099324 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 02 19:01:27.099324 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self.session.rollback() Jun 02 19:01:27.099324 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 02 19:01:27.099324 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self.force_reraise() Jun 02 19:01:27.099324 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 02 19:01:27.099324 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise self.value Jun 02 19:01:27.099324 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 02 19:01:27.099324 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 02 19:01:27.099324 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 02 19:01:27.099324 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application session.commit() Jun 02 19:01:27.099324 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit Jun 02 19:01:27.100271 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 02 19:01:27.100271 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 02 19:01:27.100271 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._prepare_impl() Jun 02 19:01:27.100271 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 02 19:01:27.100271 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self.session.flush() Jun 02 19:01:27.100271 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush Jun 02 19:01:27.100271 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._flush(objects) Jun 02 19:01:27.100271 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush Jun 02 19:01:27.100271 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 02 19:01:27.100271 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 02 19:01:27.100271 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application compat.raise_( Jun 02 19:01:27.100271 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ Jun 02 19:01:27.100271 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception Jun 02 19:01:27.100271 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush Jun 02 19:01:27.100271 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application flush_context.execute() Jun 02 19:01:27.100271 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 02 19:01:27.100271 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rec.execute(self) Jun 02 19:01:27.101332 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 02 19:01:27.101332 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 02 19:01:27.101332 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 02 19:01:27.101332 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 02 19:01:27.101332 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 02 19:01:27.101332 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 02 19:01:27.101332 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 02 19:01:27.101332 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 02 19:01:27.101332 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 02 19:01:27.101332 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 02 19:01:27.101332 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 02 19:01:27.101332 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 02 19:01:27.101332 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 02 19:01:27.101332 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 02 19:01:27.101332 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 02 19:01:27.102117 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 02 19:01:27.102117 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ Jun 02 19:01:27.102117 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception Jun 02 19:01:27.102117 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 02 19:01:27.102117 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 02 19:01:27.102117 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 02 19:01:27.102117 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 02 19:01:27.102117 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 02 19:01:27.102117 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application result = self._query(query) Jun 02 19:01:27.102117 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 02 19:01:27.102117 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application conn.query(q) Jun 02 19:01:27.102117 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 02 19:01:27.102117 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 02 19:01:27.102117 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 02 19:01:27.102117 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application result.read() Jun 02 19:01:27.102117 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 02 19:01:27.102117 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 02 19:01:27.103881 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 02 19:01:27.103881 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 02 19:01:27.103881 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 02 19:01:27.103881 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 02 19:01:27.103881 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 02 19:01:27.103881 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 02 19:01:27.103881 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 02 19:01:27.103881 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 02 19:01:27.103881 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [parameters: {'id': 'b2138ffa8fc04514b25cb811c8c46c6e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 02 19:01:27.103881 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 02 19:01:27.103881 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:27.103881 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 02 19:01:27.103881 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:27.103881 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:27.103881 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 02 19:01:27.104898 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 02 19:01:27.104898 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 02 19:01:27.104898 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 02 19:01:27.104898 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 02 19:01:27.104898 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:27.104898 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 02 19:01:27.104898 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:27.104898 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:27.104898 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:27.104898 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:27.104898 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:27.104898 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:27.104898 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:27.104898 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:27.104898 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:27.104898 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:27.104898 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:27.106233 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:27.106233 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 02 19:01:27.106233 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 02 19:01:27.106233 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:27.106233 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:27.106233 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 02 19:01:27.106233 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 02 19:01:27.106233 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b2138ffa8fc04514b25cb811c8c46c6e. Jun 02 19:01:27.106233 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:27.106233 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 474/946] 10.222.0.33 () {60 vars in 1140 bytes} [Thu Jun 2 19:01:26 2022] POST /identity/v3/projects => generated 262 bytes in 260 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 02 19:01:27.106233 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d8dea-7c60-4fd9-990a-15eec982af3a None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:27.106233 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d8dea-7c60-4fd9-990a-15eec982af3a None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:27.107100 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d8dea-7c60-4fd9-990a-15eec982af3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:27.124790 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-1b4d8dea-7c60-4fd9-990a-15eec982af3a None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:01:27.126353 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-1b4d8dea-7c60-4fd9-990a-15eec982af3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:01:27.154629 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:27.156379 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:27.205286 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c012bf41-884b-4701-b81e-4b5ecff18b69 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:27.205844 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 475/947] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:01:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 02 19:01:27.206713 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4d8dea-7c60-4fd9-990a-15eec982af3a None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:27.260957 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a786921-4ac1-4878-8ff5-1fda8ffe29c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:27.261699 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a786921-4ac1-4878-8ff5-1fda8ffe29c9 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:27.261993 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a786921-4ac1-4878-8ff5-1fda8ffe29c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:27.262271 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a786921-4ac1-4878-8ff5-1fda8ffe29c9 None admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:27.263151 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a786921-4ac1-4878-8ff5-1fda8ffe29c9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:27.263466 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a786921-4ac1-4878-8ff5-1fda8ffe29c9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:27.264964 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a786921-4ac1-4878-8ff5-1fda8ffe29c9 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:27.282978 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 473/948] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 2 19:01:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:27.282978 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a786921-4ac1-4878-8ff5-1fda8ffe29c9 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:27.334399 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4d8dea-7c60-4fd9-990a-15eec982af3a None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:27.373565 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9dfc00-a248-418d-8591-0852ce7b948b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:27.374291 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9dfc00-a248-418d-8591-0852ce7b948b None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:27.374580 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9dfc00-a248-418d-8591-0852ce7b948b None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:27.374886 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9dfc00-a248-418d-8591-0852ce7b948b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:27.382718 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9dfc00-a248-418d-8591-0852ce7b948b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:27.384613 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9dfc00-a248-418d-8591-0852ce7b948b None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:27.396140 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a786921-4ac1-4878-8ff5-1fda8ffe29c9 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:27.402971 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-6e9dfc00-a248-418d-8591-0852ce7b948b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:01:27.402971 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:27.402971 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:27.402971 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:27.402971 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:27.402971 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:27.402971 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:27.402971 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:27.402971 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:27.402971 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:27.402971 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:27.402971 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:27.402971 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 02 19:01:27.402971 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 02 19:01:27.402971 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 02 19:01:27.403870 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 02 19:01:27.403870 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:27.403870 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:27.403870 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:27.403870 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:27.403870 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:27.403870 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:27.403870 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:27.403870 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:27.403870 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:27.403870 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:27.403870 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:27.403870 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:27.403870 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:27.403870 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:27.403870 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:27.403870 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:27.404752 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 02 19:01:27.404752 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 02 19:01:27.404752 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 02 19:01:27.404752 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 02 19:01:27.404752 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 02 19:01:27.404752 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 02 19:01:27.404752 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:01:27.404752 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:27.410491 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 476/949] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 2 19:01:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:27.410491 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9dfc00-a248-418d-8591-0852ce7b948b None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:27.418674 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:27.433993 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c873b89-9e60-4073-abb0-5b1af3025f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:27.434837 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c873b89-9e60-4073-abb0-5b1af3025f2e None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:27.435192 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c873b89-9e60-4073-abb0-5b1af3025f2e None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:27.435411 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c873b89-9e60-4073-abb0-5b1af3025f2e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:27.436080 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c873b89-9e60-4073-abb0-5b1af3025f2e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:27.437605 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c873b89-9e60-4073-abb0-5b1af3025f2e None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:27.439846 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:27.450361 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-6c873b89-9e60-4073-abb0-5b1af3025f2e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 02 19:01:27.450361 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:27.450361 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 02 19:01:27.450361 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 02 19:01:27.450361 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 02 19:01:27.450361 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 02 19:01:27.450361 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 02 19:01:27.450361 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 02 19:01:27.450361 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 02 19:01:27.450361 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:01:27.450361 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 02 19:01:27.450361 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:01:27.450361 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:27.450361 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:27.450361 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:27.450361 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:27.450361 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:27.458154 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:27.458154 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:27.458154 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:27.458154 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:27.458154 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:27.458154 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:27.458154 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 02 19:01:27.458154 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 02 19:01:27.458154 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 02 19:01:27.458154 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 02 19:01:27.458154 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:27.458154 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:27.458154 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:27.458154 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:27.458154 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:27.458154 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:27.459097 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:27.459097 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:27.459097 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:27.459097 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:27.459097 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:27.459097 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:27.459097 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:27.459097 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:27.459097 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:27.459097 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:27.459097 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 02 19:01:27.459097 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 02 19:01:27.459097 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 02 19:01:27.459097 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:01:27.459097 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 474/950] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Jun 2 19:01:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:27.459971 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c873b89-9e60-4073-abb0-5b1af3025f2e None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:27.538394 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9dfc00-a248-418d-8591-0852ce7b948b None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:27.574241 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c873b89-9e60-4073-abb0-5b1af3025f2e None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:27.574549 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba11f0f-4790-4bbe-879d-c282c16e2bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:27.575231 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba11f0f-4790-4bbe-879d-c282c16e2bca None admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:27.575555 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba11f0f-4790-4bbe-879d-c282c16e2bca None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:27.575922 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba11f0f-4790-4bbe-879d-c282c16e2bca None admin] PATH_INFO: `/v3/projects/8c04020eaebd4647804cf8653f32d9e5/users/04f64813f09348eb91cb7bb2869f7612/roles/8d5938aa1af04179bc44ffc0e0f376d9` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:27.578298 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba11f0f-4790-4bbe-879d-c282c16e2bca None admin] RBAC: Authorizing `identity:create_grant(project_id=8c04020eaebd4647804cf8653f32d9e5, user_id=04f64813f09348eb91cb7bb2869f7612, role_id=8d5938aa1af04179bc44ffc0e0f376d9)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:27.579782 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba11f0f-4790-4bbe-879d-c282c16e2bca None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:27.608836 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4b7fa5-f04f-4e0b-8f63-a2c88347833a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:27.609508 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b7fa5-f04f-4e0b-8f63-a2c88347833a None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:27.609796 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b7fa5-f04f-4e0b-8f63-a2c88347833a None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:27.610076 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b7fa5-f04f-4e0b-8f63-a2c88347833a None admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:27.610870 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4b7fa5-f04f-4e0b-8f63-a2c88347833a None admin] RBAC: Adding query filter params (name=service) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:27.611224 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4b7fa5-f04f-4e0b-8f63-a2c88347833a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:27.612738 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4b7fa5-f04f-4e0b-8f63-a2c88347833a None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:27.638923 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 475/951] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Jun 2 19:01:27 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:27.638923 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4b7fa5-f04f-4e0b-8f63-a2c88347833a None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:27.683527 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 477/952] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 2 19:01:27 2022] PUT /identity/v3/projects/8c04020eaebd4647804cf8653f32d9e5/users/04f64813f09348eb91cb7bb2869f7612/roles/8d5938aa1af04179bc44ffc0e0f376d9 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:01:27.700565 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:27.772121 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4b7fa5-f04f-4e0b-8f63-a2c88347833a None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:27.815285 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4794fe76-dbca-420c-89e1-48d245fa0ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:27.815986 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794fe76-dbca-420c-89e1-48d245fa0ee1 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:27.816277 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794fe76-dbca-420c-89e1-48d245fa0ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:27.816559 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794fe76-dbca-420c-89e1-48d245fa0ee1 None admin] PATH_INFO: `/v3/services` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:27.817307 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4794fe76-dbca-420c-89e1-48d245fa0ee1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:27.817623 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4794fe76-dbca-420c-89e1-48d245fa0ee1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:27.819195 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4794fe76-dbca-420c-89e1-48d245fa0ee1 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:27.825484 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 476/953] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Jun 2 19:01:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:27.831827 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba11f0f-4790-4bbe-879d-c282c16e2bca None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:27.856193 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57c2ce5-88fe-419b-b206-845fa27bc128 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:27.856466 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57c2ce5-88fe-419b-b206-845fa27bc128 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:27.857074 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57c2ce5-88fe-419b-b206-845fa27bc128 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:27.858454 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 477/954] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:01:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:01:27.864373 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c195b3e-e555-422d-947f-5d8d77426b84 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:27.864662 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c195b3e-e555-422d-947f-5d8d77426b84 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:27.864949 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c195b3e-e555-422d-947f-5d8d77426b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:27.886482 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-1c195b3e-e555-422d-947f-5d8d77426b84 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:01:27.888766 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-1c195b3e-e555-422d-947f-5d8d77426b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:01:27.944521 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:27.970116 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba11f0f-4790-4bbe-879d-c282c16e2bca None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:27.982829 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 478/955] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:01:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 02 19:01:28.007663 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c195b3e-e555-422d-947f-5d8d77426b84 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:28.017068 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eef8d98-9c40-420d-90de-d14331afa3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:28.017752 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eef8d98-9c40-420d-90de-d14331afa3df None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:28.018058 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eef8d98-9c40-420d-90de-d14331afa3df None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:28.018326 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eef8d98-9c40-420d-90de-d14331afa3df None admin] PATH_INFO: `/v3/services` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:28.019079 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eef8d98-9c40-420d-90de-d14331afa3df None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:28.019393 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eef8d98-9c40-420d-90de-d14331afa3df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:28.020777 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eef8d98-9c40-420d-90de-d14331afa3df None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:28.029759 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 478/956] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Jun 2 19:01:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:28.036360 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eef8d98-9c40-420d-90de-d14331afa3df None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:28.055287 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:28.120788 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c195b3e-e555-422d-947f-5d8d77426b84 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:28.153611 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eef8d98-9c40-420d-90de-d14331afa3df None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:28.162088 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72041998-7523-47e9-b3ba-9c11e4f3891b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:28.163235 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72041998-7523-47e9-b3ba-9c11e4f3891b None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:28.163731 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72041998-7523-47e9-b3ba-9c11e4f3891b None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:28.164231 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72041998-7523-47e9-b3ba-9c11e4f3891b None admin] PATH_INFO: `/v3/services/compute` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:28.165638 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72041998-7523-47e9-b3ba-9c11e4f3891b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:28.167306 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72041998-7523-47e9-b3ba-9c11e4f3891b None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:28.180153 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-72041998-7523-47e9-b3ba-9c11e4f3891b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 02 19:01:28.180153 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:28.180153 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 02 19:01:28.180153 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 02 19:01:28.180153 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 02 19:01:28.180153 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 02 19:01:28.180153 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 02 19:01:28.180153 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 02 19:01:28.180153 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 02 19:01:28.180153 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:01:28.180153 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 02 19:01:28.180153 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:01:28.180153 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:28.180153 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:28.180153 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:28.180153 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:28.180153 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:28.181771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:28.181771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:28.181771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:28.181771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:28.181771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:28.181771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:28.181771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 02 19:01:28.181771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 02 19:01:28.181771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 02 19:01:28.181771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 02 19:01:28.181771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:28.181771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:28.181771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:28.181771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:28.181771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:28.181771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:28.182602 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:28.182602 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:28.182602 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:28.182602 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:28.182602 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:28.182602 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:28.182602 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:28.182602 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:28.182602 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:28.182602 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:28.182602 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 02 19:01:28.182602 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 02 19:01:28.182602 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 02 19:01:28.182602 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:01:28.185641 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 479/957] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 2 19:01:28 2022] GET /identity/v3/services/compute => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 02 19:01:28.194310 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72041998-7523-47e9-b3ba-9c11e4f3891b None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:28.194501 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194b988e-dfdd-482d-9056-31c7674d1e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:28.194501 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b988e-dfdd-482d-9056-31c7674d1e17 None admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:28.194501 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b988e-dfdd-482d-9056-31c7674d1e17 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:28.194501 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b988e-dfdd-482d-9056-31c7674d1e17 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:28.194881 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194b988e-dfdd-482d-9056-31c7674d1e17 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:28.195790 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194b988e-dfdd-482d-9056-31c7674d1e17 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:28.274077 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:28.283850 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 479/958] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Jun 2 19:01:28 2022] POST /identity/v3/endpoints => generated 349 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 02 19:01:28.307962 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72041998-7523-47e9-b3ba-9c11e4f3891b None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:28.310090 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:28.356380 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d490f06d-4da5-45c2-9a54-958970bdde1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:28.357341 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d490f06d-4da5-45c2-9a54-958970bdde1a None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:28.357810 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d490f06d-4da5-45c2-9a54-958970bdde1a None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:28.358260 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d490f06d-4da5-45c2-9a54-958970bdde1a None admin] PATH_INFO: `/v3/services` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:28.359263 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d490f06d-4da5-45c2-9a54-958970bdde1a None admin] RBAC: Adding query filter params (name=compute) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:28.359771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d490f06d-4da5-45c2-9a54-958970bdde1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:28.361385 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d490f06d-4da5-45c2-9a54-958970bdde1a None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:28.367634 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 480/959] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Jun 2 19:01:28 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:28.374298 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194b988e-dfdd-482d-9056-31c7674d1e17 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:28.491231 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194b988e-dfdd-482d-9056-31c7674d1e17 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:28.532307 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092edb31-7985-4cc2-83ac-26d0d34bc427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:28.533008 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092edb31-7985-4cc2-83ac-26d0d34bc427 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:28.533298 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092edb31-7985-4cc2-83ac-26d0d34bc427 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:28.533579 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092edb31-7985-4cc2-83ac-26d0d34bc427 None admin] PATH_INFO: `/v3/services` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:28.534308 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092edb31-7985-4cc2-83ac-26d0d34bc427 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:28.534621 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092edb31-7985-4cc2-83ac-26d0d34bc427 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:28.536048 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092edb31-7985-4cc2-83ac-26d0d34bc427 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:28.541672 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 480/960] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Jun 2 19:01:28 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:30.072006 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db292170-9f4b-4640-b099-e063310dce09 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:30.072546 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db292170-9f4b-4640-b099-e063310dce09 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:30.072546 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db292170-9f4b-4640-b099-e063310dce09 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:30.074422 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 481/961] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:01:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:01:30.080533 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3971847-2470-4404-adaa-5d2b80de6eda None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:30.081081 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3971847-2470-4404-adaa-5d2b80de6eda None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:30.081081 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3971847-2470-4404-adaa-5d2b80de6eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:30.100321 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-b3971847-2470-4404-adaa-5d2b80de6eda None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:01:30.101547 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-b3971847-2470-4404-adaa-5d2b80de6eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:01:30.133251 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dedf481-808e-4462-8258-6126b4d588a0 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:30.133577 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dedf481-808e-4462-8258-6126b4d588a0 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:30.134511 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dedf481-808e-4462-8258-6126b4d588a0 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:30.136250 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 482/962] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:01:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:01:30.142128 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eacbc2d-b699-4a40-9d2c-04af04a8d6c7 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:30.142834 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eacbc2d-b699-4a40-9d2c-04af04a8d6c7 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:30.143188 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eacbc2d-b699-4a40-9d2c-04af04a8d6c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:30.161936 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-7eacbc2d-b699-4a40-9d2c-04af04a8d6c7 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:01:30.163540 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-7eacbc2d-b699-4a40-9d2c-04af04a8d6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:01:30.186071 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:30.197399 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 481/963] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:01:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 02 19:01:30.223471 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3971847-2470-4404-adaa-5d2b80de6eda None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:30.253108 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 483/964] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:01:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 02 19:01:30.283035 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eacbc2d-b699-4a40-9d2c-04af04a8d6c7 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:30.346729 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3971847-2470-4404-adaa-5d2b80de6eda None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:30.402094 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4b20ef-3507-4a1d-9862-31ac5616c9ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:30.403122 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b20ef-3507-4a1d-9862-31ac5616c9ed None admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:30.403597 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b20ef-3507-4a1d-9862-31ac5616c9ed None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:30.404066 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b20ef-3507-4a1d-9862-31ac5616c9ed None admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:30.405350 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4b20ef-3507-4a1d-9862-31ac5616c9ed None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:30.407976 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4b20ef-3507-4a1d-9862-31ac5616c9ed None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:30.428120 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:30.438425 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eacbc2d-b699-4a40-9d2c-04af04a8d6c7 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:30.456380 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.sql.core [None req-1b4b20ef-3507-4a1d-9862-31ac5616c9ed None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 02 19:01:30.456380 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 02 19:01:30.456380 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [parameters: {'user_id': 'c8c48baacfb1417d86fc436201f133e9', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 02 19:01:30.456380 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80491) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 02 19:01:30.467208 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-1b4b20ef-3507-4a1d-9862-31ac5616c9ed None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 02 19:01:30.467208 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:30.467208 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 02 19:01:30.467208 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 02 19:01:30.467208 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 02 19:01:30.467208 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 02 19:01:30.467208 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 02 19:01:30.467208 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application result = self._query(query) Jun 02 19:01:30.467208 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 02 19:01:30.467208 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application conn.query(q) Jun 02 19:01:30.467208 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 02 19:01:30.467208 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 02 19:01:30.467208 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 02 19:01:30.467208 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application result.read() Jun 02 19:01:30.467208 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 02 19:01:30.475573 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 02 19:01:30.475573 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 02 19:01:30.475573 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 02 19:01:30.475573 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 02 19:01:30.475573 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 02 19:01:30.475573 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 02 19:01:30.475573 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 02 19:01:30.475573 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 02 19:01:30.475573 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:30.475573 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 02 19:01:30.475573 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:30.475573 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:30.475573 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 02 19:01:30.475573 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 02 19:01:30.475573 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 02 19:01:30.475573 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 02 19:01:30.475573 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 02 19:01:30.475573 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application next(self.gen) Jun 02 19:01:30.476747 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 02 19:01:30.476747 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application yield resource Jun 02 19:01:30.476747 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 02 19:01:30.476747 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application next(self.gen) Jun 02 19:01:30.476747 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 02 19:01:30.476747 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self.session.rollback() Jun 02 19:01:30.476747 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 02 19:01:30.476747 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self.force_reraise() Jun 02 19:01:30.476747 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 02 19:01:30.476747 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise self.value Jun 02 19:01:30.476747 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 02 19:01:30.476747 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 02 19:01:30.476747 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 02 19:01:30.476747 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application session.commit() Jun 02 19:01:30.476747 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit Jun 02 19:01:30.476747 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 02 19:01:30.476747 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 02 19:01:30.478372 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._prepare_impl() Jun 02 19:01:30.478372 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 02 19:01:30.478372 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self.session.flush() Jun 02 19:01:30.478372 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush Jun 02 19:01:30.478372 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._flush(objects) Jun 02 19:01:30.478372 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush Jun 02 19:01:30.478372 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 02 19:01:30.478372 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 02 19:01:30.478372 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application compat.raise_( Jun 02 19:01:30.478372 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ Jun 02 19:01:30.478372 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception Jun 02 19:01:30.478372 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush Jun 02 19:01:30.478372 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application flush_context.execute() Jun 02 19:01:30.478372 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 02 19:01:30.478372 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rec.execute(self) Jun 02 19:01:30.478372 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 02 19:01:30.478372 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 02 19:01:30.479527 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 02 19:01:30.479527 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 02 19:01:30.479527 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 02 19:01:30.479527 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 02 19:01:30.479527 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 02 19:01:30.479527 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 02 19:01:30.479527 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 02 19:01:30.479527 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 02 19:01:30.479527 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 02 19:01:30.479527 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 02 19:01:30.479527 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 02 19:01:30.479527 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 02 19:01:30.479527 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 02 19:01:30.479527 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 02 19:01:30.479527 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ Jun 02 19:01:30.479527 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception Jun 02 19:01:30.480577 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 02 19:01:30.480577 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 02 19:01:30.480577 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 02 19:01:30.480577 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 02 19:01:30.480577 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 02 19:01:30.480577 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application result = self._query(query) Jun 02 19:01:30.480577 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 02 19:01:30.480577 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application conn.query(q) Jun 02 19:01:30.480577 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 02 19:01:30.480577 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 02 19:01:30.480577 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 02 19:01:30.480577 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application result.read() Jun 02 19:01:30.480577 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 02 19:01:30.480577 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 02 19:01:30.480577 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 02 19:01:30.480577 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 02 19:01:30.481567 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 02 19:01:30.481567 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 02 19:01:30.481567 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 02 19:01:30.481567 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 02 19:01:30.481567 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 02 19:01:30.481567 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 02 19:01:30.481567 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [parameters: {'user_id': 'c8c48baacfb1417d86fc436201f133e9', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 02 19:01:30.481567 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 02 19:01:30.481567 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:30.481567 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 02 19:01:30.481567 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:30.481567 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:30.481567 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:30.481567 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:30.481567 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:30.482652 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:30.482652 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:30.482652 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:30.482652 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:30.482652 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:30.482652 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:30.482652 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:30.482652 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 02 19:01:30.482652 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 02 19:01:30.482652 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:30.482652 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:30.482652 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 02 19:01:30.482652 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:01:30.482652 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 02 19:01:30.482652 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:01:30.482652 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 02 19:01:30.482652 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 02 19:01:30.485946 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 02 19:01:30.485946 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 02 19:01:30.485946 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 02 19:01:30.485946 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 02 19:01:30.485946 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 02 19:01:30.485946 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:30.485946 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 482/965] 10.222.0.33 () {60 vars in 1131 bytes} [Thu Jun 2 19:01:30 2022] POST /identity/v3/users => generated 160 bytes in 262 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:30.488377 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4b20ef-3507-4a1d-9862-31ac5616c9ed None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:30.490304 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46765fb5-1ebf-430a-883a-a6af46ac23be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:30.491583 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46765fb5-1ebf-430a-883a-a6af46ac23be None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:30.491938 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46765fb5-1ebf-430a-883a-a6af46ac23be None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:30.492338 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46765fb5-1ebf-430a-883a-a6af46ac23be None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:30.515217 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46765fb5-1ebf-430a-883a-a6af46ac23be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:01:30.526611 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-46765fb5-1ebf-430a-883a-a6af46ac23be None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 02 19:01:30.526611 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:30.526611 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:30.526611 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:30.526611 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:30.526611 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:30.526611 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:30.526611 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:30.526611 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:30.526611 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:30.526611 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:30.526611 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:30.526611 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:01:30.526611 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:01:30.526611 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:01:30.526611 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:01:30.528544 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:01:30.528544 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:01:30.528544 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:01:30.528544 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:01:30.528544 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:30.528544 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:30.528544 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:30.528544 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:30.528544 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:30.528544 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:30.528544 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:30.528544 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:30.528544 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:30.528544 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:30.528544 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:30.528544 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:30.529632 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:30.529632 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:30.529632 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:30.529632 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:30.529632 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:01:30.529632 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:01:30.529632 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:01:30.529632 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:01:30.529632 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:01:30.529632 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:01:30.529632 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 02 19:01:30.529632 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:01:30.534016 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 484/966] 10.222.0.33 () {56 vars in 1120 bytes} [Thu Jun 2 19:01:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 02 19:01:30.535189 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46765fb5-1ebf-430a-883a-a6af46ac23be None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:30.593430 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:30.646900 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4b20ef-3507-4a1d-9862-31ac5616c9ed None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:30.657202 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46765fb5-1ebf-430a-883a-a6af46ac23be None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:30.687424 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3b3826-0f2c-4ca6-872b-7922be30edbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:30.688342 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3b3826-0f2c-4ca6-872b-7922be30edbf None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:30.688636 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3b3826-0f2c-4ca6-872b-7922be30edbf None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:30.688918 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3b3826-0f2c-4ca6-872b-7922be30edbf None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:30.698953 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f812f107-e8af-483f-9630-919ed706525c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:30.699521 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3b3826-0f2c-4ca6-872b-7922be30edbf None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:30.700254 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f812f107-e8af-483f-9630-919ed706525c None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:30.700901 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f812f107-e8af-483f-9630-919ed706525c None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:30.701210 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3b3826-0f2c-4ca6-872b-7922be30edbf None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:30.701905 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f812f107-e8af-483f-9630-919ed706525c None admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:30.703638 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f812f107-e8af-483f-9630-919ed706525c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:30.704446 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f812f107-e8af-483f-9630-919ed706525c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:30.707026 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f812f107-e8af-483f-9630-919ed706525c None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:30.717599 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-7f3b3826-0f2c-4ca6-872b-7922be30edbf None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 02 19:01:30.717599 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:30.717599 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:30.717599 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:30.717599 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:30.717599 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:30.717599 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:30.717599 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:30.717599 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:30.717599 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:30.717599 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:30.717599 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:30.717599 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 02 19:01:30.717599 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 02 19:01:30.717599 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 02 19:01:30.717599 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 02 19:01:30.718701 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:30.718701 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:30.718701 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 02 19:01:30.718701 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:01:30.718701 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 02 19:01:30.718701 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:01:30.718701 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:30.718701 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:30.718701 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:30.718701 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:30.718701 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:30.718701 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:30.718701 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:30.718701 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:30.718701 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:30.718701 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:30.718701 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:30.718701 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:30.720281 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:30.720281 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:30.720281 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 02 19:01:30.720281 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 02 19:01:30.720281 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 02 19:01:30.720281 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 02 19:01:30.720281 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 02 19:01:30.720281 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 02 19:01:30.720281 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 02 19:01:30.720281 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:30.721127 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 485/967] 10.222.0.33 () {56 vars in 1115 bytes} [Thu Jun 2 19:01:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:30.724074 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 483/968] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 2 19:01:30 2022] GET /identity/v3/users/cinder => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:30.724760 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f812f107-e8af-483f-9630-919ed706525c None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:30.730784 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3b3826-0f2c-4ca6-872b-7922be30edbf None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:30.746246 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:30.859043 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f812f107-e8af-483f-9630-919ed706525c None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:30.889150 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3b3826-0f2c-4ca6-872b-7922be30edbf None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:30.901595 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fdbe76-f186-4b44-bb4b-3990e07e7b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:30.902323 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fdbe76-f186-4b44-bb4b-3990e07e7b6c None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:30.902612 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fdbe76-f186-4b44-bb4b-3990e07e7b6c None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:30.902938 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fdbe76-f186-4b44-bb4b-3990e07e7b6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:30.912693 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:30.916125 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fdbe76-f186-4b44-bb4b-3990e07e7b6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:30.917663 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fdbe76-f186-4b44-bb4b-3990e07e7b6c None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:30.934097 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0913592a-bce5-40e4-ba71-a8e589e91b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:30.934792 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0913592a-bce5-40e4-ba71-a8e589e91b54 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:30.935083 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0913592a-bce5-40e4-ba71-a8e589e91b54 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:30.935359 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0913592a-bce5-40e4-ba71-a8e589e91b54 None admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:30.936176 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0913592a-bce5-40e4-ba71-a8e589e91b54 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:30.936483 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0913592a-bce5-40e4-ba71-a8e589e91b54 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:30.936833 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-15fdbe76-f186-4b44-bb4b-3990e07e7b6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:01:30.936833 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:30.936833 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:30.936833 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:30.936833 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:30.936833 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:30.936833 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:30.936833 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:30.936833 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:30.936833 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:30.936833 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:30.936833 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:30.936833 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:01:30.936833 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:01:30.936833 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:01:30.937773 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:01:30.937773 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:30.937773 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:30.937773 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:30.937773 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:30.937773 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:30.937773 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:30.937773 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:30.937773 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:30.937773 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:30.937773 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:30.937773 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:30.937773 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:30.937773 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:30.937773 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:30.937773 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:30.937773 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:30.938656 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:01:30.938656 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:01:30.938656 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:01:30.938656 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:01:30.938937 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0913592a-bce5-40e4-ba71-a8e589e91b54 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:30.942942 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 486/969] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:01:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:30.944322 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fcc15e-304a-4503-bdf8-020a02123033 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:30.944616 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fcc15e-304a-4503-bdf8-020a02123033 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:30.944899 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fcc15e-304a-4503-bdf8-020a02123033 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:30.946316 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 487/970] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:01:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:01:30.947492 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef2f19-90c2-4df0-9589-13d7dbc167e2 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:30.947777 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef2f19-90c2-4df0-9589-13d7dbc167e2 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:30.948053 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef2f19-90c2-4df0-9589-13d7dbc167e2 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:30.950884 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 488/971] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:01:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:01:30.950884 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aef2f19-90c2-4df0-9589-13d7dbc167e2 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:30.981314 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 484/972] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Jun 2 19:01:30 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:30.982710 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec18599-babb-46b1-a7fd-67a421a425b1 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:30.983026 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec18599-babb-46b1-a7fd-67a421a425b1 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:30.983306 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec18599-babb-46b1-a7fd-67a421a425b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:31.004176 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-8ec18599-babb-46b1-a7fd-67a421a425b1 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:01:31.009450 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-8ec18599-babb-46b1-a7fd-67a421a425b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:01:31.049186 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:31.085932 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 485/973] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:01:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 02 19:01:31.087644 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9ac41a-86bf-4a92-9ee8-0382706f0ba6 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:31.087939 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9ac41a-86bf-4a92-9ee8-0382706f0ba6 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:31.088268 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9ac41a-86bf-4a92-9ee8-0382706f0ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:31.107604 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aef2f19-90c2-4df0-9589-13d7dbc167e2 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:31.109016 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-ab9ac41a-86bf-4a92-9ee8-0382706f0ba6 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:01:31.110285 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-ab9ac41a-86bf-4a92-9ee8-0382706f0ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:01:31.168398 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40120165-c0ea-4833-9507-9af9fa9a2f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:31.169111 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40120165-c0ea-4833-9507-9af9fa9a2f42 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:31.169488 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40120165-c0ea-4833-9507-9af9fa9a2f42 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:31.169835 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40120165-c0ea-4833-9507-9af9fa9a2f42 None admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:31.170663 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40120165-c0ea-4833-9507-9af9fa9a2f42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:31.171071 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40120165-c0ea-4833-9507-9af9fa9a2f42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:31.172620 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40120165-c0ea-4833-9507-9af9fa9a2f42 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:31.193490 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 489/974] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:01:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:31.194552 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40120165-c0ea-4833-9507-9af9fa9a2f42 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:31.220385 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 486/975] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:01:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 02 19:01:31.224496 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9ac41a-86bf-4a92-9ee8-0382706f0ba6 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:31.279697 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:31.351936 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40120165-c0ea-4833-9507-9af9fa9a2f42 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:31.373428 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9ac41a-86bf-4a92-9ee8-0382706f0ba6 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:31.383983 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:31.393907 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e9b91c-4476-4e7a-a963-7f218f6c1b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:31.404864 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e9b91c-4476-4e7a-a963-7f218f6c1b52 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:31.404864 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e9b91c-4476-4e7a-a963-7f218f6c1b52 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:31.404864 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e9b91c-4476-4e7a-a963-7f218f6c1b52 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:31.404864 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e9b91c-4476-4e7a-a963-7f218f6c1b52 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:31.404864 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e9b91c-4476-4e7a-a963-7f218f6c1b52 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:31.409788 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-92e9b91c-4476-4e7a-a963-7f218f6c1b52 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 02 19:01:31.409788 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:31.409788 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 02 19:01:31.409788 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 02 19:01:31.409788 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 02 19:01:31.409788 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 02 19:01:31.409788 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 02 19:01:31.409788 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 02 19:01:31.409788 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 02 19:01:31.409788 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:01:31.409788 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 02 19:01:31.409788 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:01:31.409788 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:31.409788 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:31.409788 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:31.409788 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:31.409788 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:31.411599 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:31.411599 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:31.411599 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:31.411599 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:31.411599 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:31.411599 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:31.411599 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 02 19:01:31.411599 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 02 19:01:31.411599 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 02 19:01:31.411599 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 02 19:01:31.411599 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:31.411599 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:31.411599 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:31.411599 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:31.411599 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:31.411599 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:31.413284 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:31.413284 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:31.413284 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:31.413284 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:31.413284 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:31.413284 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:31.413284 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:31.413284 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:31.413284 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:31.413284 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:31.413284 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 02 19:01:31.413284 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 02 19:01:31.413284 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 02 19:01:31.413284 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:01:31.413284 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 490/976] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Jun 2 19:01:31 2022] GET /identity/v3/services/object-store => generated 93 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:31.415570 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e9b91c-4476-4e7a-a963-7f218f6c1b52 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:31.427103 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e61219-c617-4443-9669-2acda1fffbd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:31.427840 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e61219-c617-4443-9669-2acda1fffbd6 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:31.428123 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e61219-c617-4443-9669-2acda1fffbd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:31.428398 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e61219-c617-4443-9669-2acda1fffbd6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:31.437405 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e61219-c617-4443-9669-2acda1fffbd6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:31.438954 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e61219-c617-4443-9669-2acda1fffbd6 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:31.455905 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-33e61219-c617-4443-9669-2acda1fffbd6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 02 19:01:31.455905 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:31.455905 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:31.455905 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:31.455905 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:31.455905 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:31.455905 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:31.455905 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:31.455905 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:31.455905 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:31.455905 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:31.455905 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:31.455905 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 02 19:01:31.455905 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 02 19:01:31.455905 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 02 19:01:31.455905 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 02 19:01:31.457894 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:31.457894 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:31.457894 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 02 19:01:31.457894 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:01:31.457894 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 02 19:01:31.457894 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:01:31.457894 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:31.457894 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:31.457894 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:31.457894 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:31.457894 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:31.457894 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:31.457894 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:31.457894 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:31.457894 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:31.457894 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:31.457894 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:31.457894 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:31.459657 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:31.459657 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:31.459657 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 02 19:01:31.459657 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 02 19:01:31.459657 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 02 19:01:31.459657 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 02 19:01:31.459657 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 02 19:01:31.459657 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 02 19:01:31.459657 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 02 19:01:31.459657 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:31.459657 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 487/977] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Jun 2 19:01:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:31.459657 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e61219-c617-4443-9669-2acda1fffbd6 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:31.546098 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e9b91c-4476-4e7a-a963-7f218f6c1b52 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:31.589403 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:31.598578 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e61219-c617-4443-9669-2acda1fffbd6 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:31.599826 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce24e5-e34d-4dc7-a970-2b9ff0219b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:31.600969 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce24e5-e34d-4dc7-a970-2b9ff0219b33 None admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:31.601418 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce24e5-e34d-4dc7-a970-2b9ff0219b33 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:31.601874 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce24e5-e34d-4dc7-a970-2b9ff0219b33 None admin] PATH_INFO: `/v3/services` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:31.603482 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ce24e5-e34d-4dc7-a970-2b9ff0219b33 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:31.605765 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ce24e5-e34d-4dc7-a970-2b9ff0219b33 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:31.640780 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734c5058-9ccb-4ed2-b3fa-8d6e0b9de895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:31.641454 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734c5058-9ccb-4ed2-b3fa-8d6e0b9de895 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:31.641738 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734c5058-9ccb-4ed2-b3fa-8d6e0b9de895 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:31.642010 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734c5058-9ccb-4ed2-b3fa-8d6e0b9de895 None admin] PATH_INFO: `/v3/services` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:31.642837 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734c5058-9ccb-4ed2-b3fa-8d6e0b9de895 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:31.643098 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734c5058-9ccb-4ed2-b3fa-8d6e0b9de895 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:31.644481 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734c5058-9ccb-4ed2-b3fa-8d6e0b9de895 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:31.650408 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 488/978] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Jun 2 19:01:31 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:31.651384 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734c5058-9ccb-4ed2-b3fa-8d6e0b9de895 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:31.679693 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 491/979] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Jun 2 19:01:31 2022] POST /identity/v3/services => generated 242 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:01:31.680350 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce24e5-e34d-4dc7-a970-2b9ff0219b33 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:31.775620 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:31.807448 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734c5058-9ccb-4ed2-b3fa-8d6e0b9de895 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:31.877548 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d47814-6149-4da7-9c3b-38ea1abbe9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:31.877548 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d47814-6149-4da7-9c3b-38ea1abbe9eb None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:31.877548 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d47814-6149-4da7-9c3b-38ea1abbe9eb None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:31.877548 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d47814-6149-4da7-9c3b-38ea1abbe9eb None admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:31.878120 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d47814-6149-4da7-9c3b-38ea1abbe9eb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:31.878120 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d47814-6149-4da7-9c3b-38ea1abbe9eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:31.878120 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d47814-6149-4da7-9c3b-38ea1abbe9eb None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:31.879249 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce24e5-e34d-4dc7-a970-2b9ff0219b33 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:31.903302 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 489/980] 10.222.0.33 () {56 vars in 1149 bytes} [Thu Jun 2 19:01:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:31.904544 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d47814-6149-4da7-9c3b-38ea1abbe9eb None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:31.922430 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:31.942009 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b78b44-1b51-4957-84c7-946357a4bd4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:31.942809 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b78b44-1b51-4957-84c7-946357a4bd4d None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:31.943319 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b78b44-1b51-4957-84c7-946357a4bd4d None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:31.943711 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b78b44-1b51-4957-84c7-946357a4bd4d None admin] PATH_INFO: `/v3/services` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:31.944486 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b78b44-1b51-4957-84c7-946357a4bd4d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:31.944803 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b78b44-1b51-4957-84c7-946357a4bd4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:31.946826 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b78b44-1b51-4957-84c7-946357a4bd4d None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:31.954502 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 492/981] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Jun 2 19:01:31 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:31.963892 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b78b44-1b51-4957-84c7-946357a4bd4d None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:32.023773 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d47814-6149-4da7-9c3b-38ea1abbe9eb None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:32.061229 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b0f73-9877-4d4b-ad5c-67893430d80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:32.061890 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b0f73-9877-4d4b-ad5c-67893430d80c None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:32.062176 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b0f73-9877-4d4b-ad5c-67893430d80c None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:32.062452 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b0f73-9877-4d4b-ad5c-67893430d80c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:32.134187 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b78b44-1b51-4957-84c7-946357a4bd4d None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:32.150280 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687b0f73-9877-4d4b-ad5c-67893430d80c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:32.153077 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687b0f73-9877-4d4b-ad5c-67893430d80c None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:32.156282 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:32.173987 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:32.181248 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ec02a1-3b48-4fc4-8dbe-fd11cf27e4c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:32.182123 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec02a1-3b48-4fc4-8dbe-fd11cf27e4c3 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:32.182535 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec02a1-3b48-4fc4-8dbe-fd11cf27e4c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:32.183052 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec02a1-3b48-4fc4-8dbe-fd11cf27e4c3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:32.184365 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ec02a1-3b48-4fc4-8dbe-fd11cf27e4c3 None admin] RBAC: Adding query filter params (interface=admin, service_id=a046ad7a170942ed8d685212a3d684b5, region_id=RegionOne) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:32.184851 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ec02a1-3b48-4fc4-8dbe-fd11cf27e4c3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:32.186647 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ec02a1-3b48-4fc4-8dbe-fd11cf27e4c3 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:32.198478 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 493/982] 10.222.0.33 () {56 vars in 1249 bytes} [Thu Jun 2 19:01:31 2022] GET /identity/v3/endpoints?service_id=a046ad7a170942ed8d685212a3d684b5&interface=admin®ion_id=RegionOne => generated 193 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:01:32.199816 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ec02a1-3b48-4fc4-8dbe-fd11cf27e4c3 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:32.304921 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 490/983] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 2 19:01:31 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 02 19:01:32.306673 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b0f73-9877-4d4b-ad5c-67893430d80c None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:32.341745 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ec02a1-3b48-4fc4-8dbe-fd11cf27e4c3 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:32.397043 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e1d4c2-4461-4e5e-bb1c-a67d2bd1afe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:32.397977 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e1d4c2-4461-4e5e-bb1c-a67d2bd1afe5 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:32.398449 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e1d4c2-4461-4e5e-bb1c-a67d2bd1afe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:32.399038 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e1d4c2-4461-4e5e-bb1c-a67d2bd1afe5 None admin] PATH_INFO: `/v3/services` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:32.399900 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e1d4c2-4461-4e5e-bb1c-a67d2bd1afe5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:32.402431 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e1d4c2-4461-4e5e-bb1c-a67d2bd1afe5 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:32.414657 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 494/984] 10.222.0.33 () {56 vars in 1087 bytes} [Thu Jun 2 19:01:32 2022] GET /identity/v3/services => generated 1727 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:01:32.459352 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b0f73-9877-4d4b-ad5c-67893430d80c None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:32.481641 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:32.517768 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa7c181-78a7-412d-8c9a-b977aee197da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:32.519746 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa7c181-78a7-412d-8c9a-b977aee197da None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:32.520713 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa7c181-78a7-412d-8c9a-b977aee197da None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:32.521412 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa7c181-78a7-412d-8c9a-b977aee197da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:32.542622 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa7c181-78a7-412d-8c9a-b977aee197da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:32.545157 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa7c181-78a7-412d-8c9a-b977aee197da None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:32.577632 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-7fa7c181-78a7-412d-8c9a-b977aee197da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:01:32.577632 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:32.577632 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:32.577632 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:32.577632 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:32.577632 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:32.577632 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:32.577632 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:32.577632 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:32.577632 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:32.577632 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:32.577632 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:32.577632 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:01:32.577632 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:01:32.577632 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:01:32.578621 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:01:32.578621 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:32.578621 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:32.578621 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:32.578621 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:32.578621 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:32.578621 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:32.578621 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:32.578621 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:32.578621 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:32.578621 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:32.578621 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:32.578621 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:32.578621 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:32.578621 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:32.578621 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:32.578621 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:32.579920 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:01:32.579920 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:01:32.579920 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:01:32.579920 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:32.580426 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 491/985] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:01:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 02 19:01:32.586330 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e1d4c2-4461-4e5e-bb1c-a67d2bd1afe5 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:32.640182 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:32.705172 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e1d4c2-4461-4e5e-bb1c-a67d2bd1afe5 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:32.759594 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2286f19b-bcf0-45d8-87dd-9ceb4cafbeb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:32.760717 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2286f19b-bcf0-45d8-87dd-9ceb4cafbeb5 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:32.761212 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2286f19b-bcf0-45d8-87dd-9ceb4cafbeb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:32.761708 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2286f19b-bcf0-45d8-87dd-9ceb4cafbeb5 None admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:32.763233 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2286f19b-bcf0-45d8-87dd-9ceb4cafbeb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:32.763797 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2286f19b-bcf0-45d8-87dd-9ceb4cafbeb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:32.766395 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2286f19b-bcf0-45d8-87dd-9ceb4cafbeb5 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:32.789707 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 495/986] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:01:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:01:32.799588 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa7c181-78a7-412d-8c9a-b977aee197da None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:32.980508 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa7c181-78a7-412d-8c9a-b977aee197da None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:33.047170 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e96f8b9-72cc-4666-a257-bdffec7f6f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:33.048462 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e96f8b9-72cc-4666-a257-bdffec7f6f2f None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:33.048908 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e96f8b9-72cc-4666-a257-bdffec7f6f2f None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:33.049357 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e96f8b9-72cc-4666-a257-bdffec7f6f2f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:33.062549 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e96f8b9-72cc-4666-a257-bdffec7f6f2f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:33.065362 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e96f8b9-72cc-4666-a257-bdffec7f6f2f None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:33.094573 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-9e96f8b9-72cc-4666-a257-bdffec7f6f2f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:01:33.094573 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:33.094573 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:33.094573 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:33.094573 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:33.094573 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:33.094573 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:33.094573 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:33.094573 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:33.094573 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:33.094573 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:33.094573 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:33.094573 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 02 19:01:33.094573 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 02 19:01:33.094573 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 02 19:01:33.096413 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 02 19:01:33.096413 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:33.096413 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:33.096413 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:33.096413 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:33.096413 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:33.096413 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:33.096413 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:33.096413 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:33.096413 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:33.096413 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:33.096413 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:33.096413 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:33.096413 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:33.096413 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:33.096413 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:33.096413 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:33.098215 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 02 19:01:33.098215 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 02 19:01:33.098215 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 02 19:01:33.098215 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 02 19:01:33.098215 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 02 19:01:33.098215 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 02 19:01:33.098215 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:01:33.098215 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:33.103539 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 492/987] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 2 19:01:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:33.107199 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2286f19b-bcf0-45d8-87dd-9ceb4cafbeb5 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:33.293921 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2286f19b-bcf0-45d8-87dd-9ceb4cafbeb5 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:33.296884 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:33.356051 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a23ac31-ad48-4b70-bf45-9ccb1657f734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:33.357109 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a23ac31-ad48-4b70-bf45-9ccb1657f734 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:33.357615 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a23ac31-ad48-4b70-bf45-9ccb1657f734 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:33.358110 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a23ac31-ad48-4b70-bf45-9ccb1657f734 None admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:33.359627 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a23ac31-ad48-4b70-bf45-9ccb1657f734 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:33.360182 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a23ac31-ad48-4b70-bf45-9ccb1657f734 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:33.362844 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a23ac31-ad48-4b70-bf45-9ccb1657f734 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:33.386893 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 496/988] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 2 19:01:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:33.395851 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e96f8b9-72cc-4666-a257-bdffec7f6f2f None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:33.425284 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:33.512989 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e96f8b9-72cc-4666-a257-bdffec7f6f2f None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:33.554371 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33186a6-605d-4a0a-a4f3-8f4e9f0a6b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:33.555063 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33186a6-605d-4a0a-a4f3-8f4e9f0a6b38 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:33.555378 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33186a6-605d-4a0a-a4f3-8f4e9f0a6b38 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:33.556246 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33186a6-605d-4a0a-a4f3-8f4e9f0a6b38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:33.557080 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33186a6-605d-4a0a-a4f3-8f4e9f0a6b38 None admin] RBAC: Adding query filter params (role.id=8d5938aa1af04179bc44ffc0e0f376d9, scope.project.id=8c04020eaebd4647804cf8653f32d9e5, user.id=04f64813f09348eb91cb7bb2869f7612) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:33.557390 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33186a6-605d-4a0a-a4f3-8f4e9f0a6b38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:33.558856 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33186a6-605d-4a0a-a4f3-8f4e9f0a6b38 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:33.565683 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 493/989] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 2 19:01:33 2022] GET /identity/v3/role_assignments?user.id=04f64813f09348eb91cb7bb2869f7612&scope.project.id=8c04020eaebd4647804cf8653f32d9e5&role.id=8d5938aa1af04179bc44ffc0e0f376d9 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:01:34.494598 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4870d297-0cfd-4578-b676-6403f3b41bb6 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:34.495195 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4870d297-0cfd-4578-b676-6403f3b41bb6 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:34.495648 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4870d297-0cfd-4578-b676-6403f3b41bb6 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:34.498230 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 497/990] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:01:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:01:34.504251 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f3566d-9159-4706-bfbb-124db1e04ab1 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:34.504766 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f3566d-9159-4706-bfbb-124db1e04ab1 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:34.504985 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f3566d-9159-4706-bfbb-124db1e04ab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:34.523487 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-91f3566d-9159-4706-bfbb-124db1e04ab1 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:01:34.525253 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-91f3566d-9159-4706-bfbb-124db1e04ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:01:34.603859 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 494/991] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:01:34 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 02 19:01:34.627174 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4870d297-0cfd-4578-b676-6403f3b41bb6 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:34.756714 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4870d297-0cfd-4578-b676-6403f3b41bb6 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:34.815820 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0476a704-f8e4-4541-8359-c13db23e73f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:34.816632 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0476a704-f8e4-4541-8359-c13db23e73f9 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:34.816911 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0476a704-f8e4-4541-8359-c13db23e73f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:34.817197 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0476a704-f8e4-4541-8359-c13db23e73f9 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:34.817841 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0476a704-f8e4-4541-8359-c13db23e73f9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:34.819262 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0476a704-f8e4-4541-8359-c13db23e73f9 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:34.832468 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-0476a704-f8e4-4541-8359-c13db23e73f9 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 02 19:01:34.832468 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:34.832468 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 02 19:01:34.832468 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 02 19:01:34.832468 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 02 19:01:34.832468 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 02 19:01:34.832468 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 02 19:01:34.832468 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 02 19:01:34.832468 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 02 19:01:34.832468 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:01:34.832468 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 02 19:01:34.832468 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:01:34.832468 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:34.832468 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:34.832468 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:34.832468 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:34.832468 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:34.834229 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:34.834229 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:34.834229 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:34.834229 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:34.834229 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:34.834229 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:34.834229 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 02 19:01:34.834229 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 02 19:01:34.834229 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 02 19:01:34.834229 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 02 19:01:34.834229 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:34.834229 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:34.834229 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:34.834229 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:34.834229 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:34.834229 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:34.836524 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:34.836524 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:34.836524 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:34.836524 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:34.836524 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:34.836524 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:34.836524 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:34.836524 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:34.836524 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:34.836524 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:34.836524 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 02 19:01:34.836524 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 02 19:01:34.836524 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 02 19:01:34.836524 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:01:34.836524 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 498/992] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 2 19:01:34 2022] GET /identity/v3/services/compute => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:34.844131 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f3566d-9159-4706-bfbb-124db1e04ab1 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:34.874723 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:34.992444 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f3566d-9159-4706-bfbb-124db1e04ab1 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:35.052166 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d747a662-6ba8-45ff-bf4c-9dadb57afb82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:35.053226 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d747a662-6ba8-45ff-bf4c-9dadb57afb82 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:35.053575 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d747a662-6ba8-45ff-bf4c-9dadb57afb82 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:35.054081 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d747a662-6ba8-45ff-bf4c-9dadb57afb82 None admin] PATH_INFO: `/v3/services` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:35.055497 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d747a662-6ba8-45ff-bf4c-9dadb57afb82 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:35.056441 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d747a662-6ba8-45ff-bf4c-9dadb57afb82 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:35.059056 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d747a662-6ba8-45ff-bf4c-9dadb57afb82 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:35.073328 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 495/993] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Jun 2 19:01:34 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:35.075363 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0476a704-f8e4-4541-8359-c13db23e73f9 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:35.127397 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:35.241616 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0476a704-f8e4-4541-8359-c13db23e73f9 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:35.287270 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827149ab-928a-457f-abe1-06204a7df286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:35.288429 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827149ab-928a-457f-abe1-06204a7df286 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:35.288940 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827149ab-928a-457f-abe1-06204a7df286 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:35.289378 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827149ab-928a-457f-abe1-06204a7df286 None admin] PATH_INFO: `/v3/services` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:35.291305 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827149ab-928a-457f-abe1-06204a7df286 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:35.291863 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827149ab-928a-457f-abe1-06204a7df286 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:35.294519 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827149ab-928a-457f-abe1-06204a7df286 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:35.306576 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 499/994] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Jun 2 19:01:35 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:35.307892 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d747a662-6ba8-45ff-bf4c-9dadb57afb82 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:35.412964 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:35.437830 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d747a662-6ba8-45ff-bf4c-9dadb57afb82 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:35.475530 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082c0149-e7f5-4925-b631-ed6c84517154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:35.476202 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082c0149-e7f5-4925-b631-ed6c84517154 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:35.476489 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082c0149-e7f5-4925-b631-ed6c84517154 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:35.476771 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082c0149-e7f5-4925-b631-ed6c84517154 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:35.477603 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082c0149-e7f5-4925-b631-ed6c84517154 None admin] RBAC: Adding query filter params (interface=public, service_id=80dea99b7cbf4f1a985d2ae96c6ca2a6, region_id=RegionOne) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:35.478465 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082c0149-e7f5-4925-b631-ed6c84517154 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:35.479904 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082c0149-e7f5-4925-b631-ed6c84517154 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:35.491126 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 496/995] 10.222.0.33 () {56 vars in 1251 bytes} [Thu Jun 2 19:01:35 2022] GET /identity/v3/endpoints?service_id=80dea99b7cbf4f1a985d2ae96c6ca2a6&interface=public®ion_id=RegionOne => generated 194 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:35.494415 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827149ab-928a-457f-abe1-06204a7df286 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:35.566115 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b53db6b-3d83-41ee-9e68-ed87e54df349 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:35.566619 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b53db6b-3d83-41ee-9e68-ed87e54df349 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:35.566928 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b53db6b-3d83-41ee-9e68-ed87e54df349 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:35.568828 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 497/996] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:01:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 02 19:01:35.576413 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f739ed69-3ef2-4ee7-8e64-9a3ba514f5ef None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:35.576841 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f739ed69-3ef2-4ee7-8e64-9a3ba514f5ef None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:35.577141 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f739ed69-3ef2-4ee7-8e64-9a3ba514f5ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:35.597436 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-f739ed69-3ef2-4ee7-8e64-9a3ba514f5ef None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:01:35.598881 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-f739ed69-3ef2-4ee7-8e64-9a3ba514f5ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:01:35.599668 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:35.616247 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827149ab-928a-457f-abe1-06204a7df286 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:35.657145 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efa9193-7294-42d0-b740-9468f35675f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:35.657826 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efa9193-7294-42d0-b740-9468f35675f7 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:35.658117 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efa9193-7294-42d0-b740-9468f35675f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:35.658398 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efa9193-7294-42d0-b740-9468f35675f7 None admin] PATH_INFO: `/v3/services` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:35.659122 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efa9193-7294-42d0-b740-9468f35675f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:35.660538 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efa9193-7294-42d0-b740-9468f35675f7 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:35.668424 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 500/997] 10.222.0.33 () {56 vars in 1087 bytes} [Thu Jun 2 19:01:35 2022] GET /identity/v3/services => generated 1727 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:01:35.697424 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 498/998] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:01:35 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 02 19:01:35.722451 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efa9193-7294-42d0-b740-9468f35675f7 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:35.900786 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efa9193-7294-42d0-b740-9468f35675f7 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:35.943777 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50d1b54-2a7e-4ece-9ae1-f1aa5beb3b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:35.944633 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d1b54-2a7e-4ece-9ae1-f1aa5beb3b12 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:35.944916 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d1b54-2a7e-4ece-9ae1-f1aa5beb3b12 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:35.945189 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d1b54-2a7e-4ece-9ae1-f1aa5beb3b12 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:35.945869 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50d1b54-2a7e-4ece-9ae1-f1aa5beb3b12 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:35.946041 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c5a293-9517-474e-a3ee-7511c1007bd7 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:35.946924 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c5a293-9517-474e-a3ee-7511c1007bd7 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:35.947216 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c5a293-9517-474e-a3ee-7511c1007bd7 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:35.947904 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50d1b54-2a7e-4ece-9ae1-f1aa5beb3b12 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:35.951051 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 499/999] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:01:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:01:35.954925 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af151a54-2e91-4c38-be8c-2587b787de5a None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:35.955235 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af151a54-2e91-4c38-be8c-2587b787de5a None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:35.955488 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af151a54-2e91-4c38-be8c-2587b787de5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:35.963565 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-a50d1b54-2a7e-4ece-9ae1-f1aa5beb3b12 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 02 19:01:35.963565 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:35.963565 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 02 19:01:35.963565 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 02 19:01:35.963565 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 02 19:01:35.963565 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 02 19:01:35.963565 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 02 19:01:35.963565 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 02 19:01:35.963565 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 02 19:01:35.963565 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:01:35.963565 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 02 19:01:35.963565 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:01:35.963565 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:35.963565 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:35.963565 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:35.963565 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:35.963565 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:35.964627 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:35.964627 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:35.964627 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:35.964627 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:35.964627 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:35.964627 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:35.964627 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 02 19:01:35.964627 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 02 19:01:35.964627 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 02 19:01:35.964627 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 02 19:01:35.964627 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:35.964627 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:35.964627 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:35.964627 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:35.964627 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:35.964627 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:35.965471 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:35.965471 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:35.965471 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:35.965471 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:35.965471 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:35.965471 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:35.965471 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:35.965471 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:35.965471 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:35.965471 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:35.965471 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 02 19:01:35.965471 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 02 19:01:35.965471 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 02 19:01:35.965471 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:01:35.965471 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 501/1000] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Jun 2 19:01:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:35.971720 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50d1b54-2a7e-4ece-9ae1-f1aa5beb3b12 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:35.978317 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-af151a54-2e91-4c38-be8c-2587b787de5a None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:01:35.979984 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-af151a54-2e91-4c38-be8c-2587b787de5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:01:35.995410 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:36.070798 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 500/1001] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:01:35 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 02 19:01:36.107486 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af151a54-2e91-4c38-be8c-2587b787de5a None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:36.126364 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50d1b54-2a7e-4ece-9ae1-f1aa5beb3b12 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:36.132462 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:36.175217 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8a78be-6dda-446f-88a2-32d8ddd6eb42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:36.175822 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8a78be-6dda-446f-88a2-32d8ddd6eb42 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:36.176637 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8a78be-6dda-446f-88a2-32d8ddd6eb42 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:36.176926 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8a78be-6dda-446f-88a2-32d8ddd6eb42 None admin] PATH_INFO: `/v3/services` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:36.178112 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8a78be-6dda-446f-88a2-32d8ddd6eb42 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:36.178438 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8a78be-6dda-446f-88a2-32d8ddd6eb42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:36.180210 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8a78be-6dda-446f-88a2-32d8ddd6eb42 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:36.193145 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 502/1002] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Jun 2 19:01:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:01:36.194333 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8a78be-6dda-446f-88a2-32d8ddd6eb42 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:36.241592 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af151a54-2e91-4c38-be8c-2587b787de5a None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:36.283495 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9822c6c9-f91f-49f0-a0fc-25e612007234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:36.284316 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822c6c9-f91f-49f0-a0fc-25e612007234 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:36.284610 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822c6c9-f91f-49f0-a0fc-25e612007234 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:36.284885 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822c6c9-f91f-49f0-a0fc-25e612007234 None admin] PATH_INFO: `/v3/services/image` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:36.285579 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9822c6c9-f91f-49f0-a0fc-25e612007234 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:36.287020 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9822c6c9-f91f-49f0-a0fc-25e612007234 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:36.300345 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-9822c6c9-f91f-49f0-a0fc-25e612007234 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 02 19:01:36.300345 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:36.300345 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 02 19:01:36.300345 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 02 19:01:36.300345 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 02 19:01:36.300345 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 02 19:01:36.300345 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 02 19:01:36.300345 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 02 19:01:36.300345 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 02 19:01:36.300345 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:36.300345 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 02 19:01:36.300345 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:36.300345 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:36.300345 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:36.300345 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:36.300345 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:36.300345 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:36.301745 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:36.301745 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:36.301745 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:36.301745 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:36.301745 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:36.301745 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:36.301745 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 02 19:01:36.301745 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 02 19:01:36.301745 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 02 19:01:36.301745 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 02 19:01:36.301745 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:36.301745 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:36.301745 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:36.301745 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:36.301745 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:36.301745 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:36.303315 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:36.303315 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:36.303315 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:36.303315 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:36.303315 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:36.303315 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:36.303315 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:36.303315 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:36.303315 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:36.303315 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:36.303315 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 02 19:01:36.303315 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 02 19:01:36.303315 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 02 19:01:36.303315 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:36.303315 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 501/1003] 10.222.0.33 () {56 vars in 1105 bytes} [Thu Jun 2 19:01:36 2022] GET /identity/v3/services/image => generated 86 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:36.307892 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9822c6c9-f91f-49f0-a0fc-25e612007234 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:36.321253 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:36.356099 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8a78be-6dda-446f-88a2-32d8ddd6eb42 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:36.391212 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:36.400853 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66af88b8-20c0-43fc-a1d4-4bbe4c0dee3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:36.400853 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af88b8-20c0-43fc-a1d4-4bbe4c0dee3d None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:36.401263 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af88b8-20c0-43fc-a1d4-4bbe4c0dee3d None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:36.401401 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af88b8-20c0-43fc-a1d4-4bbe4c0dee3d None admin] PATH_INFO: `/v3/services` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:36.404017 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66af88b8-20c0-43fc-a1d4-4bbe4c0dee3d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:36.404017 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66af88b8-20c0-43fc-a1d4-4bbe4c0dee3d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:36.404017 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66af88b8-20c0-43fc-a1d4-4bbe4c0dee3d None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:36.410022 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 503/1004] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Jun 2 19:01:36 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:01:36.419009 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66af88b8-20c0-43fc-a1d4-4bbe4c0dee3d None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:36.432094 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9822c6c9-f91f-49f0-a0fc-25e612007234 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:36.488996 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e57b232-c540-4ca3-8a3f-5f488d43fe4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:36.489916 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e57b232-c540-4ca3-8a3f-5f488d43fe4f None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:36.490206 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e57b232-c540-4ca3-8a3f-5f488d43fe4f None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:36.490488 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e57b232-c540-4ca3-8a3f-5f488d43fe4f None admin] PATH_INFO: `/v3/services` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:36.491372 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e57b232-c540-4ca3-8a3f-5f488d43fe4f None admin] RBAC: Adding query filter params (name=image) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:36.491718 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e57b232-c540-4ca3-8a3f-5f488d43fe4f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:36.493334 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e57b232-c540-4ca3-8a3f-5f488d43fe4f None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:36.499300 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 502/1005] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:01:36 2022] GET /identity/v3/services?name=image => generated 122 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:01:36.505247 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e57b232-c540-4ca3-8a3f-5f488d43fe4f None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:36.557328 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66af88b8-20c0-43fc-a1d4-4bbe4c0dee3d None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:36.614076 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b65e3d-d6cc-482a-b051-4612f3ac4b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:36.615164 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b65e3d-d6cc-482a-b051-4612f3ac4b2b None admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:36.615628 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b65e3d-d6cc-482a-b051-4612f3ac4b2b None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:36.616283 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b65e3d-d6cc-482a-b051-4612f3ac4b2b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:36.617619 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b65e3d-d6cc-482a-b051-4612f3ac4b2b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:36.620573 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b65e3d-d6cc-482a-b051-4612f3ac4b2b None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:36.657625 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e57b232-c540-4ca3-8a3f-5f488d43fe4f None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:36.695476 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917494b4-e0c2-4c9e-90bb-570841bee0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:36.696079 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917494b4-e0c2-4c9e-90bb-570841bee0e6 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:36.697020 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917494b4-e0c2-4c9e-90bb-570841bee0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:36.697176 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917494b4-e0c2-4c9e-90bb-570841bee0e6 None admin] PATH_INFO: `/v3/services` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:36.697968 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917494b4-e0c2-4c9e-90bb-570841bee0e6 None admin] RBAC: Adding query filter params (type=image) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:36.698281 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917494b4-e0c2-4c9e-90bb-570841bee0e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:36.699704 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917494b4-e0c2-4c9e-90bb-570841bee0e6 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:36.701302 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:36.705465 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 503/1006] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:01:36 2022] GET /identity/v3/services?type=image => generated 122 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:01:36.732339 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 504/1007] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Jun 2 19:01:36 2022] POST /identity/v3/endpoints => generated 325 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:01:38.051273 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef458e0-6e21-4362-acb3-ce975ced8af6 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:38.052127 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef458e0-6e21-4362-acb3-ce975ced8af6 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:38.052127 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef458e0-6e21-4362-acb3-ce975ced8af6 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:38.053690 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 504/1008] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:01:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 02 19:01:38.061316 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b84d99b-186a-49cb-8819-4befe091d17f None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:38.061856 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b84d99b-186a-49cb-8819-4befe091d17f None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:38.062423 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b84d99b-186a-49cb-8819-4befe091d17f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:38.083025 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:38.091483 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-8b84d99b-186a-49cb-8819-4befe091d17f None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:01:38.093657 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-8b84d99b-186a-49cb-8819-4befe091d17f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:01:38.250727 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 505/1009] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:01:38 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 02 19:01:38.292258 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef458e0-6e21-4362-acb3-ce975ced8af6 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:38.495939 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef458e0-6e21-4362-acb3-ce975ced8af6 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:38.563965 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33059006-47ab-4c4b-8cba-833e9abdb0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:38.565358 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33059006-47ab-4c4b-8cba-833e9abdb0d2 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:38.565868 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33059006-47ab-4c4b-8cba-833e9abdb0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:38.566368 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33059006-47ab-4c4b-8cba-833e9abdb0d2 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:38.567600 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33059006-47ab-4c4b-8cba-833e9abdb0d2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:38.569072 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33059006-47ab-4c4b-8cba-833e9abdb0d2 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:38.592184 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-33059006-47ab-4c4b-8cba-833e9abdb0d2 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 02 19:01:38.592184 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:38.592184 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 02 19:01:38.592184 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 02 19:01:38.592184 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 02 19:01:38.592184 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 02 19:01:38.592184 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 02 19:01:38.592184 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 02 19:01:38.592184 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 02 19:01:38.592184 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:38.592184 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 02 19:01:38.592184 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:38.592184 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:38.592184 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:38.592184 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:38.592184 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:38.592184 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:38.594235 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:38.594235 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:38.594235 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:38.594235 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:38.594235 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:38.594235 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:38.594235 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 02 19:01:38.594235 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 02 19:01:38.594235 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 02 19:01:38.594235 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 02 19:01:38.594235 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:38.594235 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:38.594235 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:38.594235 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:38.594235 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:38.594235 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:38.596184 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:38.596184 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:38.596184 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:38.596184 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:38.596184 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:38.596184 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:38.596184 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:38.596184 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:38.596184 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:38.596184 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:38.596184 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 02 19:01:38.596184 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 02 19:01:38.596184 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 02 19:01:38.596184 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:38.596184 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 505/1010] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 2 19:01:38 2022] GET /identity/v3/services/compute => generated 88 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:38.604899 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b84d99b-186a-49cb-8819-4befe091d17f None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:38.795023 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b84d99b-186a-49cb-8819-4befe091d17f None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:38.853556 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:38.863128 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d092adba-65ba-4cf5-92d0-019062010ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:38.864338 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d092adba-65ba-4cf5-92d0-019062010ba3 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:38.864818 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d092adba-65ba-4cf5-92d0-019062010ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:38.865282 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d092adba-65ba-4cf5-92d0-019062010ba3 None admin] PATH_INFO: `/v3/services` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:38.866553 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d092adba-65ba-4cf5-92d0-019062010ba3 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:38.867079 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d092adba-65ba-4cf5-92d0-019062010ba3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:38.870134 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d092adba-65ba-4cf5-92d0-019062010ba3 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:38.884734 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 506/1011] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Jun 2 19:01:38 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:38.891595 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33059006-47ab-4c4b-8cba-833e9abdb0d2 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:39.003691 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:39.060957 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33059006-47ab-4c4b-8cba-833e9abdb0d2 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:39.079199 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c867769-5ae9-446a-b20a-9211c336c5ad None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:39.079950 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c867769-5ae9-446a-b20a-9211c336c5ad None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:39.079950 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c867769-5ae9-446a-b20a-9211c336c5ad None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:39.081404 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 507/1012] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:01:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 02 19:01:39.087188 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c437641a-2f51-4a23-a7ee-58d6df4aa897 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:39.087607 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c437641a-2f51-4a23-a7ee-58d6df4aa897 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:39.087741 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c437641a-2f51-4a23-a7ee-58d6df4aa897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:39.106646 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-c437641a-2f51-4a23-a7ee-58d6df4aa897 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:01:39.107963 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-c437641a-2f51-4a23-a7ee-58d6df4aa897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:01:39.109218 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679e7ccf-67d8-4930-ac3e-32b3d0bae504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:39.110127 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e7ccf-67d8-4930-ac3e-32b3d0bae504 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:39.110575 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e7ccf-67d8-4930-ac3e-32b3d0bae504 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:39.111037 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e7ccf-67d8-4930-ac3e-32b3d0bae504 None admin] PATH_INFO: `/v3/services` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:39.111946 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679e7ccf-67d8-4930-ac3e-32b3d0bae504 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:39.112265 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679e7ccf-67d8-4930-ac3e-32b3d0bae504 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:39.113719 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679e7ccf-67d8-4930-ac3e-32b3d0bae504 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:39.125051 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 506/1013] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Jun 2 19:01:38 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:39.133733 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679e7ccf-67d8-4930-ac3e-32b3d0bae504 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:39.197384 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 508/1014] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:01:39 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 02 19:01:39.220894 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c437641a-2f51-4a23-a7ee-58d6df4aa897 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:39.257917 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679e7ccf-67d8-4930-ac3e-32b3d0bae504 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:39.298586 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:39.311607 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8780bf0c-268e-4028-bf27-a7b998e4b40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:39.312244 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8780bf0c-268e-4028-bf27-a7b998e4b40c None admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:39.313038 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8780bf0c-268e-4028-bf27-a7b998e4b40c None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:39.313320 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8780bf0c-268e-4028-bf27-a7b998e4b40c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:39.314084 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8780bf0c-268e-4028-bf27-a7b998e4b40c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:39.315536 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8780bf0c-268e-4028-bf27-a7b998e4b40c None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:39.356484 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c437641a-2f51-4a23-a7ee-58d6df4aa897 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:39.403436 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:39.416539 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 507/1015] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Jun 2 19:01:39 2022] POST /identity/v3/endpoints => generated 334 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 02 19:01:39.417483 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709564fe-648a-4a85-b9f1-1e2e01eb864c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:39.418674 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709564fe-648a-4a85-b9f1-1e2e01eb864c None admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:39.419243 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709564fe-648a-4a85-b9f1-1e2e01eb864c None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:39.419762 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709564fe-648a-4a85-b9f1-1e2e01eb864c None admin] PATH_INFO: `/v3/services` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:39.421257 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709564fe-648a-4a85-b9f1-1e2e01eb864c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:39.424032 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709564fe-648a-4a85-b9f1-1e2e01eb864c None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:39.526793 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 509/1016] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Jun 2 19:01:39 2022] POST /identity/v3/services => generated 242 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 02 19:01:39.677351 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f4701-2025-42f7-856d-428a92f27ddc None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:39.677885 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f4701-2025-42f7-856d-428a92f27ddc None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:39.678194 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f4701-2025-42f7-856d-428a92f27ddc None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:39.680862 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 508/1017] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:01:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:01:39.691495 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f9e17c-fd96-4950-b959-d7392922da2f None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:39.692003 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f9e17c-fd96-4950-b959-d7392922da2f None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:39.692504 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f9e17c-fd96-4950-b959-d7392922da2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:39.722222 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-70f9e17c-fd96-4950-b959-d7392922da2f None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:01:39.724381 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-70f9e17c-fd96-4950-b959-d7392922da2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:01:39.854290 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 510/1018] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:01:39 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 02 19:01:39.897971 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62f4701-2025-42f7-856d-428a92f27ddc None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:40.100507 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62f4701-2025-42f7-856d-428a92f27ddc None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:40.173500 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a93a27b-8e37-44ae-ace6-7d1ba0b3c28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:40.174930 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93a27b-8e37-44ae-ace6-7d1ba0b3c28c None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:40.175438 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93a27b-8e37-44ae-ace6-7d1ba0b3c28c None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:40.175923 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93a27b-8e37-44ae-ace6-7d1ba0b3c28c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:40.178208 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a93a27b-8e37-44ae-ace6-7d1ba0b3c28c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:40.181839 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a93a27b-8e37-44ae-ace6-7d1ba0b3c28c None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:40.185114 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 509/1019] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:01:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:40.194683 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f9e17c-fd96-4950-b959-d7392922da2f None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:40.386855 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f9e17c-fd96-4950-b959-d7392922da2f None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:40.413948 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:40.454625 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36217e3-ebd4-4202-a9ad-892c8ad33d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:40.455803 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36217e3-ebd4-4202-a9ad-892c8ad33d5b None admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:40.456299 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36217e3-ebd4-4202-a9ad-892c8ad33d5b None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:40.456790 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36217e3-ebd4-4202-a9ad-892c8ad33d5b None admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:40.458097 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36217e3-ebd4-4202-a9ad-892c8ad33d5b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:40.461125 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36217e3-ebd4-4202-a9ad-892c8ad33d5b None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:40.606270 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 511/1020] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Jun 2 19:01:40 2022] POST /identity/v3/projects => generated 311 bytes in 408 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 02 19:01:41.978572 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f8ca5-52e7-4135-9a07-6e1dd7e99200 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:41.979449 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f8ca5-52e7-4135-9a07-6e1dd7e99200 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:41.979449 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f8ca5-52e7-4135-9a07-6e1dd7e99200 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:41.980825 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 510/1021] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:01:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 02 19:01:41.986470 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210bc3c3-e5a3-4a09-ba18-14ee6ac5f71c None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:41.987292 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210bc3c3-e5a3-4a09-ba18-14ee6ac5f71c None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:41.987292 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210bc3c3-e5a3-4a09-ba18-14ee6ac5f71c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:42.006436 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-210bc3c3-e5a3-4a09-ba18-14ee6ac5f71c None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:01:42.007713 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-210bc3c3-e5a3-4a09-ba18-14ee6ac5f71c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:01:42.063740 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a6d697-367b-49ae-a56f-0d97c5b21de9 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:42.064008 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a6d697-367b-49ae-a56f-0d97c5b21de9 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:42.064289 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a6d697-367b-49ae-a56f-0d97c5b21de9 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:42.066010 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 511/1022] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:01:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 02 19:01:42.073017 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843f89c3-a7df-45f4-99c1-11fd9335bdb5 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:42.073526 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843f89c3-a7df-45f4-99c1-11fd9335bdb5 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:42.073989 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843f89c3-a7df-45f4-99c1-11fd9335bdb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:42.096946 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 512/1023] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:01:41 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 02 19:01:42.097269 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-843f89c3-a7df-45f4-99c1-11fd9335bdb5 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:01:42.098577 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-843f89c3-a7df-45f4-99c1-11fd9335bdb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:01:42.140477 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210bc3c3-e5a3-4a09-ba18-14ee6ac5f71c None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:42.156006 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:42.181546 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 512/1024] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:01:42 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 02 19:01:42.220774 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843f89c3-a7df-45f4-99c1-11fd9335bdb5 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:42.259173 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:42.267772 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210bc3c3-e5a3-4a09-ba18-14ee6ac5f71c None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:42.311014 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8945941f-1420-4214-af6a-897a39475ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:42.312151 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8945941f-1420-4214-af6a-897a39475ff7 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:42.312624 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8945941f-1420-4214-af6a-897a39475ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:42.313227 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8945941f-1420-4214-af6a-897a39475ff7 None admin] PATH_INFO: `/v3/services/image` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:42.314380 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8945941f-1420-4214-af6a-897a39475ff7 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:42.316379 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8945941f-1420-4214-af6a-897a39475ff7 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:42.331562 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-8945941f-1420-4214-af6a-897a39475ff7 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 02 19:01:42.331562 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:42.331562 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 02 19:01:42.331562 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 02 19:01:42.331562 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 02 19:01:42.331562 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 02 19:01:42.331562 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 02 19:01:42.331562 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 02 19:01:42.331562 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 02 19:01:42.331562 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:01:42.331562 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 02 19:01:42.331562 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:01:42.331562 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:42.331562 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:42.331562 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:42.331562 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:42.331562 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:42.332787 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:42.332787 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:42.332787 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:42.332787 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:42.332787 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:42.332787 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:42.332787 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 02 19:01:42.332787 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 02 19:01:42.332787 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 02 19:01:42.332787 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 02 19:01:42.332787 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:42.332787 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:42.332787 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:42.332787 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:42.332787 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:42.332787 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:42.333778 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:42.333778 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:42.333778 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:42.333778 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:42.333778 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:42.333778 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:42.333778 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:42.333778 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:42.333778 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:42.333778 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:42.333778 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 02 19:01:42.333778 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 02 19:01:42.333778 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 02 19:01:42.333778 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:01:42.336419 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 513/1025] 10.222.0.33 () {56 vars in 1105 bytes} [Thu Jun 2 19:01:42 2022] GET /identity/v3/services/image => generated 86 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:42.342991 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8945941f-1420-4214-af6a-897a39475ff7 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:42.346380 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843f89c3-a7df-45f4-99c1-11fd9335bdb5 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:42.409078 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b52544-8546-4a05-85bf-bc29bc6706e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:42.410242 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b52544-8546-4a05-85bf-bc29bc6706e6 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:42.410811 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b52544-8546-4a05-85bf-bc29bc6706e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:42.411343 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b52544-8546-4a05-85bf-bc29bc6706e6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:42.427011 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b52544-8546-4a05-85bf-bc29bc6706e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:01:42.435746 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-a1b52544-8546-4a05-85bf-bc29bc6706e6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 02 19:01:42.435746 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:42.435746 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:42.435746 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:42.435746 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:42.435746 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:42.435746 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:42.435746 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:42.435746 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:42.435746 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:42.435746 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:42.435746 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:42.435746 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:01:42.435746 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:01:42.435746 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:01:42.435746 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:01:42.436975 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:01:42.436975 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:01:42.436975 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:01:42.436975 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:01:42.436975 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:42.436975 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:42.436975 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:42.436975 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:42.436975 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:42.436975 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:42.436975 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:42.436975 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:42.436975 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:42.436975 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:42.436975 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:42.436975 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:42.438028 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:42.438028 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:42.438028 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:42.438028 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:42.438028 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:01:42.438028 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:01:42.438028 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:01:42.438028 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:01:42.438028 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:01:42.438028 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:01:42.438028 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 02 19:01:42.438028 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:42.445265 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 513/1026] 10.222.0.33 () {56 vars in 1120 bytes} [Thu Jun 2 19:01:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:42.446471 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b52544-8546-4a05-85bf-bc29bc6706e6 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:42.497085 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8945941f-1420-4214-af6a-897a39475ff7 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:42.519234 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:42.553673 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6211ba-516b-4201-9749-b3a9f9cd4aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:42.554742 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6211ba-516b-4201-9749-b3a9f9cd4aa2 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:42.555235 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6211ba-516b-4201-9749-b3a9f9cd4aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:42.555701 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6211ba-516b-4201-9749-b3a9f9cd4aa2 None admin] PATH_INFO: `/v3/services` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:42.557546 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6211ba-516b-4201-9749-b3a9f9cd4aa2 None admin] RBAC: Adding query filter params (name=image) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:42.558065 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6211ba-516b-4201-9749-b3a9f9cd4aa2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:42.560590 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6211ba-516b-4201-9749-b3a9f9cd4aa2 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:42.569711 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 514/1027] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:01:42 2022] GET /identity/v3/services?name=image => generated 122 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:01:42.576180 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6211ba-516b-4201-9749-b3a9f9cd4aa2 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:42.620283 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:42.626041 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b52544-8546-4a05-85bf-bc29bc6706e6 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:42.669357 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5226c1c8-49ee-4861-8580-f0bb5d629cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:42.670341 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5226c1c8-49ee-4861-8580-f0bb5d629cdc None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:42.670827 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5226c1c8-49ee-4861-8580-f0bb5d629cdc None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:42.671251 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5226c1c8-49ee-4861-8580-f0bb5d629cdc None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:42.672207 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5226c1c8-49ee-4861-8580-f0bb5d629cdc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:42.672703 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5226c1c8-49ee-4861-8580-f0bb5d629cdc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:42.674290 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5226c1c8-49ee-4861-8580-f0bb5d629cdc None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:42.684204 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 514/1028] 10.222.0.33 () {56 vars in 1115 bytes} [Thu Jun 2 19:01:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:42.692934 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5226c1c8-49ee-4861-8580-f0bb5d629cdc None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:42.768332 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6211ba-516b-4201-9749-b3a9f9cd4aa2 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:42.824157 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e328277-51a2-4d6f-81e9-0654412fe635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:42.825074 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e328277-51a2-4d6f-81e9-0654412fe635 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:42.825502 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e328277-51a2-4d6f-81e9-0654412fe635 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:42.825970 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e328277-51a2-4d6f-81e9-0654412fe635 None admin] PATH_INFO: `/v3/services` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:42.826984 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e328277-51a2-4d6f-81e9-0654412fe635 None admin] RBAC: Adding query filter params (type=image) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:42.827476 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e328277-51a2-4d6f-81e9-0654412fe635 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:42.829135 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e328277-51a2-4d6f-81e9-0654412fe635 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:42.835936 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 515/1029] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:01:42 2022] GET /identity/v3/services?type=image => generated 350 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:42.839414 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5226c1c8-49ee-4861-8580-f0bb5d629cdc None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:42.842380 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:42.849206 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e328277-51a2-4d6f-81e9-0654412fe635 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:42.904046 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618fec1f-8124-48d5-89f8-38776d031e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:42.904847 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618fec1f-8124-48d5-89f8-38776d031e78 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:42.905134 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618fec1f-8124-48d5-89f8-38776d031e78 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:42.905820 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618fec1f-8124-48d5-89f8-38776d031e78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:42.921575 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618fec1f-8124-48d5-89f8-38776d031e78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:42.923124 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618fec1f-8124-48d5-89f8-38776d031e78 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:42.943241 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-618fec1f-8124-48d5-89f8-38776d031e78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:01:42.943241 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:42.943241 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:42.943241 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:42.943241 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:42.943241 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:42.943241 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:42.943241 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:42.943241 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:42.943241 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:42.943241 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:42.943241 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:42.943241 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:01:42.943241 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:01:42.943241 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:01:42.945211 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:01:42.945211 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:42.945211 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:42.945211 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:42.945211 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:42.945211 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:42.945211 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:42.945211 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:42.945211 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:42.945211 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:42.945211 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:42.945211 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:42.945211 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:42.945211 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:42.945211 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:42.945211 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:42.945211 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:42.947809 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:01:42.947809 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:01:42.947809 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:01:42.947809 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:42.947809 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 515/1030] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:01:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 02 19:01:42.954895 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618fec1f-8124-48d5-89f8-38776d031e78 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:42.997060 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:43.024014 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e328277-51a2-4d6f-81e9-0654412fe635 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:43.087996 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eee9243-5fce-41eb-a67a-b28619bdd8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:43.089000 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eee9243-5fce-41eb-a67a-b28619bdd8ce None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:43.089491 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eee9243-5fce-41eb-a67a-b28619bdd8ce None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:43.089943 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eee9243-5fce-41eb-a67a-b28619bdd8ce None admin] PATH_INFO: `/v3/endpoints` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:43.091447 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eee9243-5fce-41eb-a67a-b28619bdd8ce None admin] RBAC: Adding query filter params (interface=public, service_id=0d8eb8dfbde14d42af330626913826ef, region_id=RegionOne) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:43.092018 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eee9243-5fce-41eb-a67a-b28619bdd8ce None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:43.094473 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eee9243-5fce-41eb-a67a-b28619bdd8ce None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:43.109035 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 516/1031] 10.222.0.33 () {56 vars in 1251 bytes} [Thu Jun 2 19:01:42 2022] GET /identity/v3/endpoints?service_id=0d8eb8dfbde14d42af330626913826ef&interface=public®ion_id=RegionOne => generated 194 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:01:43.114256 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eee9243-5fce-41eb-a67a-b28619bdd8ce None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:43.127208 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618fec1f-8124-48d5-89f8-38776d031e78 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:43.195511 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b66007-ff63-4c0d-9310-98d88b6d6f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:43.196226 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b66007-ff63-4c0d-9310-98d88b6d6f02 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:43.196578 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b66007-ff63-4c0d-9310-98d88b6d6f02 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:43.196866 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b66007-ff63-4c0d-9310-98d88b6d6f02 None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:43.197770 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b66007-ff63-4c0d-9310-98d88b6d6f02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:43.198108 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b66007-ff63-4c0d-9310-98d88b6d6f02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:43.199817 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b66007-ff63-4c0d-9310-98d88b6d6f02 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:43.217481 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 516/1032] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:01:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:01:43.224109 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b66007-ff63-4c0d-9310-98d88b6d6f02 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:43.245574 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:43.302938 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eee9243-5fce-41eb-a67a-b28619bdd8ce None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:43.368236 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c64e2ee-df76-4b8d-8be8-29a0b3982dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:43.369801 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c64e2ee-df76-4b8d-8be8-29a0b3982dad None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:43.370262 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c64e2ee-df76-4b8d-8be8-29a0b3982dad None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:43.370718 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c64e2ee-df76-4b8d-8be8-29a0b3982dad None admin] PATH_INFO: `/v3/services` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:43.372002 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c64e2ee-df76-4b8d-8be8-29a0b3982dad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:43.374117 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c64e2ee-df76-4b8d-8be8-29a0b3982dad None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:43.387662 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 517/1033] 10.222.0.33 () {56 vars in 1087 bytes} [Thu Jun 2 19:01:43 2022] GET /identity/v3/services => generated 1957 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 02 19:01:43.402468 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b66007-ff63-4c0d-9310-98d88b6d6f02 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:43.467491 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d14c11-7783-427a-90df-b585668740f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:43.468798 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d14c11-7783-427a-90df-b585668740f5 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:43.469243 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d14c11-7783-427a-90df-b585668740f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:43.469689 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d14c11-7783-427a-90df-b585668740f5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:43.484250 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d14c11-7783-427a-90df-b585668740f5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:43.486800 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d14c11-7783-427a-90df-b585668740f5 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:43.513044 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-90d14c11-7783-427a-90df-b585668740f5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 02 19:01:43.513044 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:43.513044 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:43.513044 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:43.513044 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:43.513044 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:43.513044 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:43.513044 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:43.513044 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:43.513044 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:43.513044 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:43.513044 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:43.513044 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 02 19:01:43.513044 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 02 19:01:43.513044 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 02 19:01:43.513044 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 02 19:01:43.514860 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:43.514860 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:43.514860 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 02 19:01:43.514860 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:01:43.514860 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 02 19:01:43.514860 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:01:43.514860 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:43.514860 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:43.514860 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:43.514860 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:43.514860 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:43.514860 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:43.514860 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:43.514860 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:43.514860 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:43.514860 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:43.514860 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:43.514860 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:43.517229 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:43.517229 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:43.517229 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 02 19:01:43.517229 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 02 19:01:43.517229 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 02 19:01:43.517229 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 02 19:01:43.517229 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 02 19:01:43.517229 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 02 19:01:43.517229 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 02 19:01:43.517229 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:43.517229 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 517/1034] 10.222.0.33 () {56 vars in 1093 bytes} [Thu Jun 2 19:01:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:43.521675 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c64e2ee-df76-4b8d-8be8-29a0b3982dad None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:43.543950 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:43.654459 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c64e2ee-df76-4b8d-8be8-29a0b3982dad None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:43.706288 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b291b3-68b9-4c2d-85d1-5d978a140f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:43.707492 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b291b3-68b9-4c2d-85d1-5d978a140f09 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:43.707989 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b291b3-68b9-4c2d-85d1-5d978a140f09 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:43.708482 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b291b3-68b9-4c2d-85d1-5d978a140f09 None admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:43.710127 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b291b3-68b9-4c2d-85d1-5d978a140f09 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:43.710680 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b291b3-68b9-4c2d-85d1-5d978a140f09 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:43.714206 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b291b3-68b9-4c2d-85d1-5d978a140f09 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:43.726910 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f30cfcb-5da3-4af0-8306-3518f62d6706 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:43.727212 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f30cfcb-5da3-4af0-8306-3518f62d6706 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:43.727486 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f30cfcb-5da3-4af0-8306-3518f62d6706 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:43.729497 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 518/1035] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:01:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 02 19:01:43.735503 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e634e54-0692-4dd1-bfc6-d551d82f3a56 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:43.735773 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e634e54-0692-4dd1-bfc6-d551d82f3a56 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:43.736061 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e634e54-0692-4dd1-bfc6-d551d82f3a56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:43.748813 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 518/1036] 10.222.0.33 () {56 vars in 1133 bytes} [Thu Jun 2 19:01:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:43.758057 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b291b3-68b9-4c2d-85d1-5d978a140f09 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:43.761560 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-6e634e54-0692-4dd1-bfc6-d551d82f3a56 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:01:43.763633 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-6e634e54-0692-4dd1-bfc6-d551d82f3a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:01:43.886593 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:43.887951 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 519/1037] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:01:43 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 02 19:01:43.913733 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e634e54-0692-4dd1-bfc6-d551d82f3a56 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:43.917694 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:43.951437 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b291b3-68b9-4c2d-85d1-5d978a140f09 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:43.994554 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8e9ac2-6afa-4e3b-81d7-eb8f4c000a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:43.995256 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8e9ac2-6afa-4e3b-81d7-eb8f4c000a1c None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:43.995545 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8e9ac2-6afa-4e3b-81d7-eb8f4c000a1c None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:43.995825 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8e9ac2-6afa-4e3b-81d7-eb8f4c000a1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:44.081317 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8e9ac2-6afa-4e3b-81d7-eb8f4c000a1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:44.082316 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e634e54-0692-4dd1-bfc6-d551d82f3a56 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:44.082847 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8e9ac2-6afa-4e3b-81d7-eb8f4c000a1c None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:44.127633 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558fa639-b943-4aa7-95fe-c6e9695097a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:44.128457 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558fa639-b943-4aa7-95fe-c6e9695097a1 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:44.128737 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558fa639-b943-4aa7-95fe-c6e9695097a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:44.129003 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558fa639-b943-4aa7-95fe-c6e9695097a1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:44.130258 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558fa639-b943-4aa7-95fe-c6e9695097a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:44.132036 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558fa639-b943-4aa7-95fe-c6e9695097a1 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:44.134617 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 520/1038] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:01:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:01:44.140271 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558fa639-b943-4aa7-95fe-c6e9695097a1 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:44.183340 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:44.230896 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 519/1039] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 2 19:01:43 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 02 19:01:44.238674 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:44.239963 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8e9ac2-6afa-4e3b-81d7-eb8f4c000a1c None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:44.289356 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558fa639-b943-4aa7-95fe-c6e9695097a1 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:44.346348 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180297a9-f63b-4b16-a2ea-4fa0fbc6d85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:44.347410 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180297a9-f63b-4b16-a2ea-4fa0fbc6d85a None admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:44.347879 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180297a9-f63b-4b16-a2ea-4fa0fbc6d85a None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:44.348336 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180297a9-f63b-4b16-a2ea-4fa0fbc6d85a None admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:44.349274 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180297a9-f63b-4b16-a2ea-4fa0fbc6d85a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:44.351821 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180297a9-f63b-4b16-a2ea-4fa0fbc6d85a None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:44.395205 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8e9ac2-6afa-4e3b-81d7-eb8f4c000a1c None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:44.432311 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a0ed4a-7372-4365-8823-bfd6673a57c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:44.433284 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a0ed4a-7372-4365-8823-bfd6673a57c9 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:44.433756 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a0ed4a-7372-4365-8823-bfd6673a57c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:44.434090 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a0ed4a-7372-4365-8823-bfd6673a57c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:44.445730 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a0ed4a-7372-4365-8823-bfd6673a57c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:44.447832 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a0ed4a-7372-4365-8823-bfd6673a57c9 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:44.458217 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 521/1040] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 2 19:01:44 2022] POST /identity/v3/users => generated 285 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 02 19:01:44.476820 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-c5a0ed4a-7372-4365-8823-bfd6673a57c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:01:44.476820 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:44.476820 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:44.476820 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:44.476820 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:44.476820 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:44.476820 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:44.476820 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:44.476820 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:44.476820 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:44.476820 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:44.476820 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:44.476820 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:01:44.476820 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:01:44.476820 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:01:44.479609 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:01:44.479609 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:44.479609 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:44.479609 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:44.479609 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:44.479609 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:44.479609 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:44.479609 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:44.479609 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:44.479609 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:44.479609 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:44.479609 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:44.479609 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:44.479609 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:44.479609 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:44.479609 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:44.479609 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:44.480852 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:01:44.480852 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:01:44.480852 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:01:44.480852 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:01:44.480852 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 520/1041] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:01:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 02 19:01:44.485693 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180297a9-f63b-4b16-a2ea-4fa0fbc6d85a None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:44.671796 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180297a9-f63b-4b16-a2ea-4fa0fbc6d85a None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:44.718981 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:44.729687 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfda569-7a97-40dd-aa3c-1b58eb7b78d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:44.730384 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfda569-7a97-40dd-aa3c-1b58eb7b78d4 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:44.730668 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfda569-7a97-40dd-aa3c-1b58eb7b78d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:44.734632 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfda569-7a97-40dd-aa3c-1b58eb7b78d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:44.734632 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfda569-7a97-40dd-aa3c-1b58eb7b78d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:44.734632 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfda569-7a97-40dd-aa3c-1b58eb7b78d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:44.734632 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfda569-7a97-40dd-aa3c-1b58eb7b78d4 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:44.750515 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 522/1042] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:01:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:01:44.758518 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a0ed4a-7372-4365-8823-bfd6673a57c9 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:44.805183 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:44.948864 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a0ed4a-7372-4365-8823-bfd6673a57c9 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:45.017834 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac444211-bf15-42fd-b3ee-bd2f85f09659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:45.019157 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac444211-bf15-42fd-b3ee-bd2f85f09659 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:45.019665 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac444211-bf15-42fd-b3ee-bd2f85f09659 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:45.020325 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac444211-bf15-42fd-b3ee-bd2f85f09659 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:45.034878 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac444211-bf15-42fd-b3ee-bd2f85f09659 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:45.037957 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac444211-bf15-42fd-b3ee-bd2f85f09659 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:45.066367 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-ac444211-bf15-42fd-b3ee-bd2f85f09659 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:01:45.066367 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:45.066367 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:45.066367 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:45.066367 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:45.066367 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:45.066367 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:45.066367 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:45.066367 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:45.066367 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:45.066367 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:45.066367 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:45.066367 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 02 19:01:45.066367 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 02 19:01:45.066367 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 02 19:01:45.068347 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 02 19:01:45.068347 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:45.068347 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:45.068347 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:45.068347 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:45.068347 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:45.068347 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:45.068347 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:45.068347 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:45.068347 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:45.068347 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:45.068347 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:45.068347 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:45.068347 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:45.068347 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:45.068347 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:45.068347 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:45.070201 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 02 19:01:45.070201 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 02 19:01:45.070201 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 02 19:01:45.070201 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 02 19:01:45.070201 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 02 19:01:45.070201 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 02 19:01:45.070201 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:01:45.070201 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:01:45.070201 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 521/1043] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 2 19:01:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 02 19:01:45.075963 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfda569-7a97-40dd-aa3c-1b58eb7b78d4 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:45.259017 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfda569-7a97-40dd-aa3c-1b58eb7b78d4 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:45.303129 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab43562f-c939-4e98-881a-4a34eeacd8d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:45.303913 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab43562f-c939-4e98-881a-4a34eeacd8d1 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:45.304338 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab43562f-c939-4e98-881a-4a34eeacd8d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:45.304740 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab43562f-c939-4e98-881a-4a34eeacd8d1 None admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:45.305709 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab43562f-c939-4e98-881a-4a34eeacd8d1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:45.306668 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab43562f-c939-4e98-881a-4a34eeacd8d1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:45.308384 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab43562f-c939-4e98-881a-4a34eeacd8d1 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:45.323998 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 523/1044] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 2 19:01:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:01:45.330891 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac444211-bf15-42fd-b3ee-bd2f85f09659 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:45.492998 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:45.513455 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac444211-bf15-42fd-b3ee-bd2f85f09659 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:45.556124 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c5e90f-751d-41dc-b2ad-50544647d087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:45.557253 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c5e90f-751d-41dc-b2ad-50544647d087 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:45.557534 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c5e90f-751d-41dc-b2ad-50544647d087 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:45.557807 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c5e90f-751d-41dc-b2ad-50544647d087 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:45.558669 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c5e90f-751d-41dc-b2ad-50544647d087 None admin] RBAC: Adding query filter params (role.id=8d5938aa1af04179bc44ffc0e0f376d9, scope.project.id=8c04020eaebd4647804cf8653f32d9e5, user.id=70664d5cbdee47ee89c120761939cc75) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:45.559019 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c5e90f-751d-41dc-b2ad-50544647d087 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:45.560498 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c5e90f-751d-41dc-b2ad-50544647d087 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:45.567557 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 522/1045] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 2 19:01:45 2022] GET /identity/v3/role_assignments?user.id=70664d5cbdee47ee89c120761939cc75&scope.project.id=8c04020eaebd4647804cf8653f32d9e5&role.id=8d5938aa1af04179bc44ffc0e0f376d9 => generated 259 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:01:46.653042 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c5ba79-8b15-4e10-a995-71d2c1bb9e47 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:46.653580 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c5ba79-8b15-4e10-a995-71d2c1bb9e47 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:46.654039 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c5ba79-8b15-4e10-a995-71d2c1bb9e47 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:46.656881 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 524/1046] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:01:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:01:46.666145 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033ab74-4bb3-45ff-929a-bfe1e47bbd59 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:46.666723 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033ab74-4bb3-45ff-929a-bfe1e47bbd59 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:46.667195 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033ab74-4bb3-45ff-929a-bfe1e47bbd59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:46.695462 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-3033ab74-4bb3-45ff-929a-bfe1e47bbd59 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:01:46.697589 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-3033ab74-4bb3-45ff-929a-bfe1e47bbd59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:01:46.795084 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 523/1047] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:01:46 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 02 19:01:46.833562 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c5ba79-8b15-4e10-a995-71d2c1bb9e47 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:46.931862 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:47.033464 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c5ba79-8b15-4e10-a995-71d2c1bb9e47 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:47.098421 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16048e0f-affb-4511-bba4-97399e334b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:47.099872 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16048e0f-affb-4511-bba4-97399e334b05 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:47.100379 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16048e0f-affb-4511-bba4-97399e334b05 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:47.100865 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16048e0f-affb-4511-bba4-97399e334b05 None admin] PATH_INFO: `/v3/services/image` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:47.102233 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16048e0f-affb-4511-bba4-97399e334b05 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:47.105239 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16048e0f-affb-4511-bba4-97399e334b05 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:47.137280 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-16048e0f-affb-4511-bba4-97399e334b05 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 02 19:01:47.137280 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:47.137280 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 02 19:01:47.137280 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 02 19:01:47.137280 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 02 19:01:47.137280 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 02 19:01:47.137280 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 02 19:01:47.137280 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 02 19:01:47.137280 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 02 19:01:47.137280 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:47.137280 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 02 19:01:47.137280 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:47.137280 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:47.137280 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:47.137280 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:47.137280 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:47.137280 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:47.139316 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:47.139316 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:47.139316 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:47.139316 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:47.139316 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:47.139316 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:47.139316 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 02 19:01:47.139316 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 02 19:01:47.139316 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 02 19:01:47.139316 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 02 19:01:47.139316 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:47.139316 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:47.139316 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:47.139316 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:47.139316 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:47.139316 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:47.140928 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:47.140928 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:47.140928 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:47.140928 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:47.140928 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:47.140928 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:47.140928 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:47.140928 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:47.140928 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:47.140928 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:47.140928 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 02 19:01:47.140928 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 02 19:01:47.140928 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 02 19:01:47.140928 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:47.143608 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 525/1048] 10.222.0.33 () {56 vars in 1105 bytes} [Thu Jun 2 19:01:46 2022] GET /identity/v3/services/image => generated 86 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 02 19:01:47.149982 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3033ab74-4bb3-45ff-929a-bfe1e47bbd59 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:47.310984 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:47.357040 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3033ab74-4bb3-45ff-929a-bfe1e47bbd59 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:47.426665 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a23169-1a30-4522-94fb-5de5efdf63eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:47.427886 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a23169-1a30-4522-94fb-5de5efdf63eb None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:47.428381 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a23169-1a30-4522-94fb-5de5efdf63eb None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:47.428870 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a23169-1a30-4522-94fb-5de5efdf63eb None admin] PATH_INFO: `/v3/services` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:47.430223 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a23169-1a30-4522-94fb-5de5efdf63eb None admin] RBAC: Adding query filter params (name=image) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:47.430798 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a23169-1a30-4522-94fb-5de5efdf63eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:47.433993 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a23169-1a30-4522-94fb-5de5efdf63eb None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:47.451256 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 524/1049] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:01:47 2022] GET /identity/v3/services?name=image => generated 122 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:47.454141 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16048e0f-affb-4511-bba4-97399e334b05 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:47.653849 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16048e0f-affb-4511-bba4-97399e334b05 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:47.677881 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:47.722527 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281974cf-ed9e-44be-9b96-6ca6bfa25a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:47.723727 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281974cf-ed9e-44be-9b96-6ca6bfa25a7f None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:47.724239 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281974cf-ed9e-44be-9b96-6ca6bfa25a7f None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:47.724716 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281974cf-ed9e-44be-9b96-6ca6bfa25a7f None admin] PATH_INFO: `/v3/services` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:47.726081 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281974cf-ed9e-44be-9b96-6ca6bfa25a7f None admin] RBAC: Adding query filter params (type=image) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:47.726646 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281974cf-ed9e-44be-9b96-6ca6bfa25a7f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:47.729448 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281974cf-ed9e-44be-9b96-6ca6bfa25a7f None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:47.747128 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 526/1050] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:01:47 2022] GET /identity/v3/services?type=image => generated 350 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:47.751081 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a23169-1a30-4522-94fb-5de5efdf63eb None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:47.862366 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8642c2d7-d91c-4e52-a6d5-4241f437c748 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:47.862366 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8642c2d7-d91c-4e52-a6d5-4241f437c748 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:47.862366 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8642c2d7-d91c-4e52-a6d5-4241f437c748 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:47.863006 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 527/1051] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:01:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 02 19:01:47.868467 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea8cac7-ed5b-4df6-ba2f-9ecaa4e8ac71 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:47.869324 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea8cac7-ed5b-4df6-ba2f-9ecaa4e8ac71 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:47.869611 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea8cac7-ed5b-4df6-ba2f-9ecaa4e8ac71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:47.888877 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-fea8cac7-ed5b-4df6-ba2f-9ecaa4e8ac71 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:01:47.890089 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-fea8cac7-ed5b-4df6-ba2f-9ecaa4e8ac71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:01:47.893750 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a23169-1a30-4522-94fb-5de5efdf63eb None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:47.938889 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b59866c-f500-4d45-8087-739e4c98d158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:47.939486 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b59866c-f500-4d45-8087-739e4c98d158 None admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:47.939762 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b59866c-f500-4d45-8087-739e4c98d158 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:47.940039 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b59866c-f500-4d45-8087-739e4c98d158 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:47.940813 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b59866c-f500-4d45-8087-739e4c98d158 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:47.942221 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b59866c-f500-4d45-8087-739e4c98d158 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:47.973644 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 528/1052] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:01:47 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 02 19:01:47.998692 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea8cac7-ed5b-4df6-ba2f-9ecaa4e8ac71 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:48.023320 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:48.042787 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 525/1053] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Jun 2 19:01:47 2022] POST /identity/v3/endpoints => generated 327 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 02 19:01:48.148866 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea8cac7-ed5b-4df6-ba2f-9ecaa4e8ac71 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:48.158928 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:48.202713 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef68c11-660d-4752-8dfd-e095bef7e8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:48.203738 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef68c11-660d-4752-8dfd-e095bef7e8ed None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:48.204158 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef68c11-660d-4752-8dfd-e095bef7e8ed None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:48.204564 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef68c11-660d-4752-8dfd-e095bef7e8ed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:48.218693 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef68c11-660d-4752-8dfd-e095bef7e8ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:01:48.227361 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-8ef68c11-660d-4752-8dfd-e095bef7e8ed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 02 19:01:48.227361 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:48.227361 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:48.227361 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:48.227361 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:48.227361 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:48.227361 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:48.227361 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:48.227361 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:48.227361 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:48.227361 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:48.227361 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:48.227361 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:01:48.227361 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:01:48.227361 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:01:48.227361 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:01:48.229412 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:01:48.229412 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:01:48.229412 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:01:48.229412 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:01:48.229412 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:48.229412 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:48.229412 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:48.229412 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:48.229412 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:48.229412 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:48.229412 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:48.229412 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:48.229412 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:48.229412 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:48.229412 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:48.229412 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:48.231953 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:48.231953 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:48.231953 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:48.231953 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:48.231953 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:01:48.231953 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:01:48.231953 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:01:48.231953 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:01:48.231953 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:01:48.231953 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:01:48.231953 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 02 19:01:48.231953 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:48.235219 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 529/1054] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 2 19:01:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 02 19:01:48.238376 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b59866c-f500-4d45-8087-739e4c98d158 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:48.395711 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b59866c-f500-4d45-8087-739e4c98d158 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:48.457870 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca1d129-8597-4426-b5ba-9a6616705737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:48.458666 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca1d129-8597-4426-b5ba-9a6616705737 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:48.459184 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca1d129-8597-4426-b5ba-9a6616705737 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:48.459583 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca1d129-8597-4426-b5ba-9a6616705737 None admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:48.460560 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca1d129-8597-4426-b5ba-9a6616705737 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:48.461004 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca1d129-8597-4426-b5ba-9a6616705737 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:48.462659 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca1d129-8597-4426-b5ba-9a6616705737 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:48.473176 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 526/1055] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 2 19:01:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:01:48.479860 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef68c11-660d-4752-8dfd-e095bef7e8ed None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:48.602228 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef68c11-660d-4752-8dfd-e095bef7e8ed None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:48.645837 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ecf045-9bcb-4ffc-8d87-4c1a4cbe1e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:48.646993 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ecf045-9bcb-4ffc-8d87-4c1a4cbe1e60 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:48.647407 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ecf045-9bcb-4ffc-8d87-4c1a4cbe1e60 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:48.647877 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ecf045-9bcb-4ffc-8d87-4c1a4cbe1e60 None admin] PATH_INFO: `/v3/users/3b414e92fbf34dd19ff334cac4e4effe` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:48.666375 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ecf045-9bcb-4ffc-8d87-4c1a4cbe1e60 None admin] RBAC: Authorizing `identity:get_user(user_id=3b414e92fbf34dd19ff334cac4e4effe)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:48.668415 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ecf045-9bcb-4ffc-8d87-4c1a4cbe1e60 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:48.670648 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 530/1056] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 2 19:01:48 2022] GET /identity/v3/users/3b414e92fbf34dd19ff334cac4e4effe => generated 285 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:48.677590 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca1d129-8597-4426-b5ba-9a6616705737 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:48.753205 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:48.806231 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca1d129-8597-4426-b5ba-9a6616705737 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:48.847309 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f1838-b9bf-434e-b3a6-651428e584c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:48.848228 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f1838-b9bf-434e-b3a6-651428e584c7 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:48.848637 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f1838-b9bf-434e-b3a6-651428e584c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:48.849050 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f1838-b9bf-434e-b3a6-651428e584c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:48.864907 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131fbb69-08f3-465a-be6e-6cfd8c6b654e None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:48.865347 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131fbb69-08f3-465a-be6e-6cfd8c6b654e None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:48.865720 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131fbb69-08f3-465a-be6e-6cfd8c6b654e None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:48.867359 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 531/1057] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:01:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 02 19:01:48.876802 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06442b5e-bd8e-4a62-9cde-49e565f09c65 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:48.877301 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06442b5e-bd8e-4a62-9cde-49e565f09c65 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:48.877799 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06442b5e-bd8e-4a62-9cde-49e565f09c65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:48.906673 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-06442b5e-bd8e-4a62-9cde-49e565f09c65 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:01:48.908843 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-06442b5e-bd8e-4a62-9cde-49e565f09c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:01:48.936791 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:48.950581 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18f1838-b9bf-434e-b3a6-651428e584c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:48.953399 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18f1838-b9bf-434e-b3a6-651428e584c7 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:49.034143 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 532/1058] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:01:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 02 19:01:49.044266 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:49.062940 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06442b5e-bd8e-4a62-9cde-49e565f09c65 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:49.129522 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 527/1059] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 2 19:01:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 02 19:01:49.139989 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f1838-b9bf-434e-b3a6-651428e584c7 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:49.216995 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06442b5e-bd8e-4a62-9cde-49e565f09c65 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:49.274170 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535f38e2-d520-48ce-91a1-ecd83c1d029b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:49.275740 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f38e2-d520-48ce-91a1-ecd83c1d029b None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:49.276526 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f38e2-d520-48ce-91a1-ecd83c1d029b None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:49.276896 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f38e2-d520-48ce-91a1-ecd83c1d029b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:49.291311 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f1838-b9bf-434e-b3a6-651428e584c7 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:49.297672 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535f38e2-d520-48ce-91a1-ecd83c1d029b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:01:49.314470 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-535f38e2-d520-48ce-91a1-ecd83c1d029b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 02 19:01:49.314470 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:49.314470 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:49.314470 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:49.314470 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:49.314470 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:49.314470 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:49.314470 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:49.314470 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:49.314470 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:49.314470 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:49.314470 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:49.314470 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:01:49.314470 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:01:49.314470 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:01:49.314470 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:01:49.315909 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:01:49.315909 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:01:49.315909 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:01:49.315909 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:01:49.315909 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:49.315909 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:49.315909 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:49.315909 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:49.315909 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:49.315909 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:49.315909 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:49.315909 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:49.315909 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:49.315909 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:49.315909 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:49.315909 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:49.319820 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:49.319820 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:49.319820 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:49.319820 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:49.319820 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:01:49.319820 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:01:49.319820 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:01:49.319820 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:01:49.319820 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:01:49.319820 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:01:49.319820 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 02 19:01:49.319820 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:49.319820 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 533/1060] 10.222.0.33 () {56 vars in 1120 bytes} [Thu Jun 2 19:01:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 02 19:01:49.324394 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535f38e2-d520-48ce-91a1-ecd83c1d029b None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:49.340573 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:49.357806 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c636091-bb60-4a6a-bf70-11b81d8424d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:49.359020 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c636091-bb60-4a6a-bf70-11b81d8424d6 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:49.359507 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c636091-bb60-4a6a-bf70-11b81d8424d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:49.359882 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c636091-bb60-4a6a-bf70-11b81d8424d6 None admin] PATH_INFO: `/v3/projects/f3f51e06fcfe468188953e05a231ce9d` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:49.362577 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c636091-bb60-4a6a-bf70-11b81d8424d6 None admin] RBAC: Authorizing `identity:get_project(project_id=f3f51e06fcfe468188953e05a231ce9d)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:49.365223 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c636091-bb60-4a6a-bf70-11b81d8424d6 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:49.374218 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 528/1061] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 2 19:01:49 2022] GET /identity/v3/projects/f3f51e06fcfe468188953e05a231ce9d => generated 311 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:49.378734 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c636091-bb60-4a6a-bf70-11b81d8424d6 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:49.434643 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:49.480941 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535f38e2-d520-48ce-91a1-ecd83c1d029b None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:49.537653 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c636091-bb60-4a6a-bf70-11b81d8424d6 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:49.547542 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf42435-15ea-4d35-a4aa-9a783d8b1827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:49.548630 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf42435-15ea-4d35-a4aa-9a783d8b1827 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:49.549144 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf42435-15ea-4d35-a4aa-9a783d8b1827 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:49.549657 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf42435-15ea-4d35-a4aa-9a783d8b1827 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:49.551184 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf42435-15ea-4d35-a4aa-9a783d8b1827 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:49.551747 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf42435-15ea-4d35-a4aa-9a783d8b1827 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:49.554436 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf42435-15ea-4d35-a4aa-9a783d8b1827 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:49.570262 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 534/1062] 10.222.0.33 () {56 vars in 1115 bytes} [Thu Jun 2 19:01:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:01:49.580838 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf42435-15ea-4d35-a4aa-9a783d8b1827 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:49.594653 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9055fb86-df2a-43a6-8f9e-206ae693dede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:49.596110 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9055fb86-df2a-43a6-8f9e-206ae693dede None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:49.596731 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9055fb86-df2a-43a6-8f9e-206ae693dede None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:49.597118 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9055fb86-df2a-43a6-8f9e-206ae693dede None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:49.598589 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9055fb86-df2a-43a6-8f9e-206ae693dede None admin] RBAC: Adding query filter params (role.id=f13fdd0847924014abf1e95594195e78, scope.project.id=f3f51e06fcfe468188953e05a231ce9d, user.id=3b414e92fbf34dd19ff334cac4e4effe) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:49.599137 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9055fb86-df2a-43a6-8f9e-206ae693dede None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:49.601331 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9055fb86-df2a-43a6-8f9e-206ae693dede None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:49.622789 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 529/1063] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 2 19:01:49 2022] GET /identity/v3/role_assignments?user.id=3b414e92fbf34dd19ff334cac4e4effe&scope.project.id=f3f51e06fcfe468188953e05a231ce9d&role.id=f13fdd0847924014abf1e95594195e78 => generated 259 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:49.693209 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:49.737135 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf42435-15ea-4d35-a4aa-9a783d8b1827 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:49.780895 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f34fb8-4472-4ae8-a3a6-7e03b3c25d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:49.781731 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f34fb8-4472-4ae8-a3a6-7e03b3c25d4d None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:49.782013 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f34fb8-4472-4ae8-a3a6-7e03b3c25d4d None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:49.782359 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f34fb8-4472-4ae8-a3a6-7e03b3c25d4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:49.796632 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f34fb8-4472-4ae8-a3a6-7e03b3c25d4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:49.798379 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f34fb8-4472-4ae8-a3a6-7e03b3c25d4d None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:49.818440 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-e4f34fb8-4472-4ae8-a3a6-7e03b3c25d4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:01:49.818440 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:49.818440 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:49.818440 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:49.818440 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:49.818440 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:49.818440 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:49.818440 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:49.818440 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:49.818440 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:49.818440 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:49.818440 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:49.818440 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:01:49.818440 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:01:49.818440 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:01:49.820376 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:01:49.820376 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:49.820376 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:49.820376 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:49.820376 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:49.820376 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:49.820376 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:49.820376 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:49.820376 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:49.820376 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:49.820376 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:49.820376 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:49.820376 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:49.820376 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:49.820376 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:49.820376 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:49.820376 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:49.821789 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:01:49.821789 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:01:49.821789 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:01:49.821789 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:49.821789 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 535/1064] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:01:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 02 19:01:49.831263 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9055fb86-df2a-43a6-8f9e-206ae693dede None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:50.029660 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:50.041190 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9055fb86-df2a-43a6-8f9e-206ae693dede None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:50.082956 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf5661-4f3c-4ae1-b077-85952c9c3aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:50.083892 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf5661-4f3c-4ae1-b077-85952c9c3aa3 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:50.084320 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf5661-4f3c-4ae1-b077-85952c9c3aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:50.084735 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf5661-4f3c-4ae1-b077-85952c9c3aa3 None admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:50.085611 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdf5661-4f3c-4ae1-b077-85952c9c3aa3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:50.086057 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdf5661-4f3c-4ae1-b077-85952c9c3aa3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:50.088215 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdf5661-4f3c-4ae1-b077-85952c9c3aa3 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:50.101920 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 530/1065] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:01:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:50.111014 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f34fb8-4472-4ae8-a3a6-7e03b3c25d4d None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:50.313299 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f34fb8-4472-4ae8-a3a6-7e03b3c25d4d None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:50.357660 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:50.371427 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df913f05-63f5-4a9a-9f32-139e9c6a72d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:50.372561 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df913f05-63f5-4a9a-9f32-139e9c6a72d1 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:50.372991 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df913f05-63f5-4a9a-9f32-139e9c6a72d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:50.373411 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df913f05-63f5-4a9a-9f32-139e9c6a72d1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:50.383275 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df913f05-63f5-4a9a-9f32-139e9c6a72d1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:50.385145 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df913f05-63f5-4a9a-9f32-139e9c6a72d1 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:50.402329 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-df913f05-63f5-4a9a-9f32-139e9c6a72d1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 02 19:01:50.402329 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:50.402329 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:50.402329 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:50.402329 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:50.402329 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:50.402329 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:50.402329 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:50.402329 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:50.402329 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:50.402329 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:50.402329 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:50.402329 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 02 19:01:50.402329 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 02 19:01:50.402329 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 02 19:01:50.402329 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 02 19:01:50.403521 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:50.403521 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:50.403521 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 02 19:01:50.403521 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:01:50.403521 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 02 19:01:50.403521 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:01:50.403521 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:50.403521 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:50.403521 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:50.403521 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:50.403521 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:50.403521 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:50.403521 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:50.403521 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:50.403521 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:50.403521 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:50.403521 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:50.403521 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:50.404461 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:50.404461 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:50.404461 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 02 19:01:50.404461 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 02 19:01:50.404461 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 02 19:01:50.404461 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 02 19:01:50.404461 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 02 19:01:50.404461 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 02 19:01:50.404461 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 02 19:01:50.404461 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:50.406925 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 536/1066] 10.222.0.33 () {56 vars in 1093 bytes} [Thu Jun 2 19:01:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 02 19:01:50.415188 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf5661-4f3c-4ae1-b077-85952c9c3aa3 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:50.497262 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d2470-bfa7-468f-a77c-15ba2d83a274 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:50.497708 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d2470-bfa7-468f-a77c-15ba2d83a274 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:50.498158 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d2470-bfa7-468f-a77c-15ba2d83a274 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:50.500275 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 537/1067] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:01:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 02 19:01:50.506636 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b86784-466a-4e15-87f5-6d5d49796f40 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:50.507099 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b86784-466a-4e15-87f5-6d5d49796f40 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:50.507280 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b86784-466a-4e15-87f5-6d5d49796f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:50.526106 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-09b86784-466a-4e15-87f5-6d5d49796f40 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:01:50.527424 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-09b86784-466a-4e15-87f5-6d5d49796f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:01:50.549485 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf5661-4f3c-4ae1-b077-85952c9c3aa3 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:50.600832 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6ff201-4d96-4259-ab75-7b9093fb4606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:50.601837 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ff201-4d96-4259-ab75-7b9093fb4606 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:50.602109 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ff201-4d96-4259-ab75-7b9093fb4606 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:50.602358 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ff201-4d96-4259-ab75-7b9093fb4606 None admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:50.604313 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6ff201-4d96-4259-ab75-7b9093fb4606 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:50.604635 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6ff201-4d96-4259-ab75-7b9093fb4606 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:50.606107 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6ff201-4d96-4259-ab75-7b9093fb4606 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:50.625793 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 538/1068] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:01:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 02 19:01:50.634086 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 531/1069] 10.222.0.33 () {56 vars in 1133 bytes} [Thu Jun 2 19:01:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:01:50.643859 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b86784-466a-4e15-87f5-6d5d49796f40 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:50.660576 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6ff201-4d96-4259-ab75-7b9093fb4606 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:50.681351 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:50.801148 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b86784-466a-4e15-87f5-6d5d49796f40 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:50.823275 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:50.842543 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6ff201-4d96-4259-ab75-7b9093fb4606 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:50.861243 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09446936-73aa-4005-bc3a-7dc22262232e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:50.862709 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09446936-73aa-4005-bc3a-7dc22262232e None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:50.863261 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09446936-73aa-4005-bc3a-7dc22262232e None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:50.863762 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09446936-73aa-4005-bc3a-7dc22262232e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:50.883387 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09446936-73aa-4005-bc3a-7dc22262232e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:50.886631 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09446936-73aa-4005-bc3a-7dc22262232e None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:50.890794 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3650528-527f-4955-ab32-5c081e4630f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:50.891829 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3650528-527f-4955-ab32-5c081e4630f9 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:50.892259 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3650528-527f-4955-ab32-5c081e4630f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:50.892661 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3650528-527f-4955-ab32-5c081e4630f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:50.919027 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-09446936-73aa-4005-bc3a-7dc22262232e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:01:50.919027 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:50.919027 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:50.919027 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:50.919027 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:50.919027 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:50.919027 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:50.919027 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:50.919027 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:50.919027 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:50.919027 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:50.919027 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:50.919027 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:01:50.919027 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:01:50.919027 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:01:50.920066 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:01:50.920066 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:50.920066 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:50.920066 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:50.920066 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:50.920066 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:50.920066 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:50.920066 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:50.920066 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:50.920066 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:50.920066 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:50.920066 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:50.920066 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:50.920066 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:50.920066 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:50.920066 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:50.920066 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:50.920962 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:01:50.920962 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:01:50.920962 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:01:50.920962 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:50.922222 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 539/1070] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:01:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:50.937710 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09446936-73aa-4005-bc3a-7dc22262232e None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:50.995625 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3650528-527f-4955-ab32-5c081e4630f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:50.997697 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3650528-527f-4955-ab32-5c081e4630f9 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:51.017176 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:51.105703 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09446936-73aa-4005-bc3a-7dc22262232e None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:51.166906 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 532/1071] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 2 19:01:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 02 19:01:51.171791 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3650528-527f-4955-ab32-5c081e4630f9 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:51.175505 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e047f6a-90c7-44e1-97a9-b47473cdf60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:51.176702 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e047f6a-90c7-44e1-97a9-b47473cdf60f None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:51.177208 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e047f6a-90c7-44e1-97a9-b47473cdf60f None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:51.177705 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e047f6a-90c7-44e1-97a9-b47473cdf60f None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:51.179176 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e047f6a-90c7-44e1-97a9-b47473cdf60f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:51.179732 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e047f6a-90c7-44e1-97a9-b47473cdf60f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:51.182594 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e047f6a-90c7-44e1-97a9-b47473cdf60f None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:51.210810 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 540/1072] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:01:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:01:51.216981 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e047f6a-90c7-44e1-97a9-b47473cdf60f None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:51.267280 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:51.354407 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3650528-527f-4955-ab32-5c081e4630f9 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:51.370830 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:51.379805 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e047f6a-90c7-44e1-97a9-b47473cdf60f None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:51.412823 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81c97df-397a-4e26-8705-a8d6466dba3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:51.414149 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81c97df-397a-4e26-8705-a8d6466dba3d None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:51.414742 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81c97df-397a-4e26-8705-a8d6466dba3d None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:51.415261 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81c97df-397a-4e26-8705-a8d6466dba3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:51.424703 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b11426-92df-4266-b130-e109899cc27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:51.425680 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b11426-92df-4266-b130-e109899cc27a None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:51.426189 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b11426-92df-4266-b130-e109899cc27a None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:51.426639 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b11426-92df-4266-b130-e109899cc27a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:51.436923 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b11426-92df-4266-b130-e109899cc27a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:51.437337 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81c97df-397a-4e26-8705-a8d6466dba3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:51.438783 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b11426-92df-4266-b130-e109899cc27a None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:51.439893 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81c97df-397a-4e26-8705-a8d6466dba3d None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:51.456512 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-96b11426-92df-4266-b130-e109899cc27a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:01:51.456512 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:51.456512 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:51.456512 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:51.456512 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:51.456512 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:51.456512 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:51.456512 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:51.456512 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:51.456512 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:51.456512 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:51.456512 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:51.456512 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 02 19:01:51.456512 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 02 19:01:51.456512 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 02 19:01:51.458381 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 02 19:01:51.458381 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:51.458381 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:51.458381 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:51.458381 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:51.458381 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:51.458381 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:51.458381 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:51.458381 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:51.458381 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:51.458381 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:51.458381 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:51.458381 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:51.458381 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:51.458381 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:51.458381 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:51.458381 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:51.460183 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 02 19:01:51.460183 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 02 19:01:51.460183 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 02 19:01:51.460183 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 02 19:01:51.460183 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 02 19:01:51.460183 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 02 19:01:51.460183 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:01:51.460183 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:51.460183 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 541/1073] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 2 19:01:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 02 19:01:51.468446 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b11426-92df-4266-b130-e109899cc27a None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:51.471855 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-c81c97df-397a-4e26-8705-a8d6466dba3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:01:51.471855 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:51.471855 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:51.471855 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:51.471855 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:51.471855 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:51.471855 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:51.471855 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:51.471855 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:51.471855 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:51.471855 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:51.471855 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:51.471855 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:01:51.471855 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:01:51.471855 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:01:51.474988 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:01:51.474988 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:51.474988 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:51.474988 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:51.474988 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:51.474988 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:51.474988 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:51.474988 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:51.474988 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:51.474988 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:51.474988 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:51.474988 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:51.474988 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:51.474988 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:51.474988 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:51.474988 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:51.474988 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:51.479367 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:01:51.479367 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:01:51.479367 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:01:51.479367 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:01:51.479367 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 533/1074] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:01:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 02 19:01:51.480378 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81c97df-397a-4e26-8705-a8d6466dba3d None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:51.638184 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:51.644145 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81c97df-397a-4e26-8705-a8d6466dba3d None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:51.672381 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b11426-92df-4266-b130-e109899cc27a None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:51.691177 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8664bcde-2dbb-4582-90af-cde0d436d78e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:51.691753 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8664bcde-2dbb-4582-90af-cde0d436d78e None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:51.692038 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8664bcde-2dbb-4582-90af-cde0d436d78e None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:51.692316 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8664bcde-2dbb-4582-90af-cde0d436d78e None admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:51.693064 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8664bcde-2dbb-4582-90af-cde0d436d78e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:51.693374 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8664bcde-2dbb-4582-90af-cde0d436d78e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:51.694849 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8664bcde-2dbb-4582-90af-cde0d436d78e None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:51.708993 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 534/1075] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:01:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:51.751119 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa137f2-51ac-47de-9cd9-8c497aa6b29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:51.752788 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa137f2-51ac-47de-9cd9-8c497aa6b29f None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:51.753579 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa137f2-51ac-47de-9cd9-8c497aa6b29f None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:51.754361 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa137f2-51ac-47de-9cd9-8c497aa6b29f None admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:51.756652 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa137f2-51ac-47de-9cd9-8c497aa6b29f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:51.757595 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa137f2-51ac-47de-9cd9-8c497aa6b29f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:51.760926 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa137f2-51ac-47de-9cd9-8c497aa6b29f None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:51.788854 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 542/1076] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 2 19:01:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:01:51.799207 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8664bcde-2dbb-4582-90af-cde0d436d78e None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:51.853032 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:52.008978 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8664bcde-2dbb-4582-90af-cde0d436d78e None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:52.071588 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85885cc2-a9c0-4a29-a0f5-ebecf485e278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:52.072634 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85885cc2-a9c0-4a29-a0f5-ebecf485e278 None admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:52.072992 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85885cc2-a9c0-4a29-a0f5-ebecf485e278 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:52.073340 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85885cc2-a9c0-4a29-a0f5-ebecf485e278 None admin] PATH_INFO: `/v3/projects/8c04020eaebd4647804cf8653f32d9e5/users/70664d5cbdee47ee89c120761939cc75/roles/8d5938aa1af04179bc44ffc0e0f376d9` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:52.076948 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85885cc2-a9c0-4a29-a0f5-ebecf485e278 None admin] RBAC: Authorizing `identity:create_grant(project_id=8c04020eaebd4647804cf8653f32d9e5, user_id=70664d5cbdee47ee89c120761939cc75, role_id=8d5938aa1af04179bc44ffc0e0f376d9)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:52.079124 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85885cc2-a9c0-4a29-a0f5-ebecf485e278 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:52.172011 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 535/1077] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 2 19:01:51 2022] PUT /identity/v3/projects/8c04020eaebd4647804cf8653f32d9e5/users/70664d5cbdee47ee89c120761939cc75/roles/8d5938aa1af04179bc44ffc0e0f376d9 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:01:52.719080 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58487977-e649-4343-95b5-2e86dfd3829c None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:52.719623 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58487977-e649-4343-95b5-2e86dfd3829c None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:52.719623 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58487977-e649-4343-95b5-2e86dfd3829c None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:52.721035 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 543/1078] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:01:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:01:52.726789 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f21976-95f2-4591-9462-bce2cbce8341 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:52.727227 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f21976-95f2-4591-9462-bce2cbce8341 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:52.727603 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f21976-95f2-4591-9462-bce2cbce8341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:52.746059 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-79f21976-95f2-4591-9462-bce2cbce8341 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:01:52.747296 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-79f21976-95f2-4591-9462-bce2cbce8341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:01:52.824374 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 536/1079] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:01:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 02 19:01:52.847857 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58487977-e649-4343-95b5-2e86dfd3829c None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:52.965738 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58487977-e649-4343-95b5-2e86dfd3829c None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:53.008643 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8947c361-e06b-4847-9573-e55438337f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:53.009925 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8947c361-e06b-4847-9573-e55438337f35 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:53.010218 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8947c361-e06b-4847-9573-e55438337f35 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:53.010498 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8947c361-e06b-4847-9573-e55438337f35 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:53.023965 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8947c361-e06b-4847-9573-e55438337f35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:01:53.032516 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-8947c361-e06b-4847-9573-e55438337f35 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 02 19:01:53.032516 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:53.032516 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:53.032516 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:53.032516 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:53.032516 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:53.032516 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:53.032516 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:53.032516 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:53.032516 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:53.032516 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:53.032516 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:53.032516 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:01:53.032516 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:01:53.032516 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:01:53.032516 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:01:53.033830 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:01:53.033830 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:01:53.033830 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:01:53.033830 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:01:53.033830 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:53.033830 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:53.033830 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:53.033830 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:53.033830 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:53.033830 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:53.033830 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:53.033830 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:53.033830 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:53.033830 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:53.033830 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:53.033830 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:53.034797 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:53.034797 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:53.034797 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:53.034797 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:53.034797 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:01:53.034797 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:01:53.034797 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:01:53.034797 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:01:53.034797 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:01:53.034797 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:01:53.034797 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 02 19:01:53.034797 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:53.034797 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 544/1080] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 2 19:01:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:53.041590 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f21976-95f2-4591-9462-bce2cbce8341 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:53.068348 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:53.213095 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f21976-95f2-4591-9462-bce2cbce8341 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:53.256636 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff8028f-e0b3-4732-bdf7-defe4262a64b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:53.257324 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff8028f-e0b3-4732-bdf7-defe4262a64b None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:53.257606 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff8028f-e0b3-4732-bdf7-defe4262a64b None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:53.257892 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff8028f-e0b3-4732-bdf7-defe4262a64b None admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:53.258703 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff8028f-e0b3-4732-bdf7-defe4262a64b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:53.258984 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff8028f-e0b3-4732-bdf7-defe4262a64b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:53.260413 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff8028f-e0b3-4732-bdf7-defe4262a64b None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:53.269321 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 537/1081] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 2 19:01:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:53.276905 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8947c361-e06b-4847-9573-e55438337f35 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:53.292809 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:53.403442 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8947c361-e06b-4847-9573-e55438337f35 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:53.441282 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a41d68f-fd34-41d6-a331-6139a1158709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:53.442164 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a41d68f-fd34-41d6-a331-6139a1158709 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:53.442457 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a41d68f-fd34-41d6-a331-6139a1158709 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:53.442735 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a41d68f-fd34-41d6-a331-6139a1158709 None admin] PATH_INFO: `/v3/users/3b414e92fbf34dd19ff334cac4e4effe` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:53.444028 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a41d68f-fd34-41d6-a331-6139a1158709 None admin] RBAC: Authorizing `identity:get_user(user_id=3b414e92fbf34dd19ff334cac4e4effe)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:53.445489 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a41d68f-fd34-41d6-a331-6139a1158709 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:53.447995 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 545/1082] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 2 19:01:53 2022] GET /identity/v3/users/3b414e92fbf34dd19ff334cac4e4effe => generated 285 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:53.455284 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff8028f-e0b3-4732-bdf7-defe4262a64b None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:53.570628 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:53.598977 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff8028f-e0b3-4732-bdf7-defe4262a64b None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:53.636330 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557c24c0-84e9-451a-bbb2-5269df7f9fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:53.637235 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557c24c0-84e9-451a-bbb2-5269df7f9fa1 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:53.637518 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557c24c0-84e9-451a-bbb2-5269df7f9fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:53.637796 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557c24c0-84e9-451a-bbb2-5269df7f9fa1 None admin] PATH_INFO: `/v3/projects/f3f51e06fcfe468188953e05a231ce9d` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:53.639045 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557c24c0-84e9-451a-bbb2-5269df7f9fa1 None admin] RBAC: Authorizing `identity:get_project(project_id=f3f51e06fcfe468188953e05a231ce9d)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:53.640493 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557c24c0-84e9-451a-bbb2-5269df7f9fa1 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:53.642217 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 538/1083] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 2 19:01:53 2022] GET /identity/v3/projects/f3f51e06fcfe468188953e05a231ce9d => generated 311 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:53.648123 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a41d68f-fd34-41d6-a331-6139a1158709 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:53.727454 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:53.753605 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a41d68f-fd34-41d6-a331-6139a1158709 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:53.790867 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64aa5ee-3a3d-4e1e-91ec-2e19a096ea38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:53.791778 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64aa5ee-3a3d-4e1e-91ec-2e19a096ea38 None admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:53.792064 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64aa5ee-3a3d-4e1e-91ec-2e19a096ea38 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:53.792341 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64aa5ee-3a3d-4e1e-91ec-2e19a096ea38 None admin] PATH_INFO: `/v3/projects/f3f51e06fcfe468188953e05a231ce9d/users/3b414e92fbf34dd19ff334cac4e4effe/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:53.794403 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64aa5ee-3a3d-4e1e-91ec-2e19a096ea38 None admin] RBAC: Authorizing `identity:create_grant(project_id=f3f51e06fcfe468188953e05a231ce9d, user_id=3b414e92fbf34dd19ff334cac4e4effe, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:53.795884 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64aa5ee-3a3d-4e1e-91ec-2e19a096ea38 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:53.891318 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 546/1084] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 2 19:01:53 2022] PUT /identity/v3/projects/f3f51e06fcfe468188953e05a231ce9d/users/3b414e92fbf34dd19ff334cac4e4effe/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:01:54.586782 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d891f7-2a8b-45db-b59b-7f1a8868bd37 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:54.587629 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d891f7-2a8b-45db-b59b-7f1a8868bd37 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:54.587629 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d891f7-2a8b-45db-b59b-7f1a8868bd37 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:54.589195 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 539/1085] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:01:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 02 19:01:54.595158 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa40af6-aebf-4bc2-bf30-0188d7ccaaf0 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:54.595454 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa40af6-aebf-4bc2-bf30-0188d7ccaaf0 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:54.595742 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa40af6-aebf-4bc2-bf30-0188d7ccaaf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:54.622957 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-aaa40af6-aebf-4bc2-bf30-0188d7ccaaf0 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:01:54.624838 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-aaa40af6-aebf-4bc2-bf30-0188d7ccaaf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:01:54.738624 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 547/1086] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:01:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 02 19:01:54.763276 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d891f7-2a8b-45db-b59b-7f1a8868bd37 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:54.880955 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d891f7-2a8b-45db-b59b-7f1a8868bd37 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:54.921553 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4f8d56-e125-42b3-be4e-a7e9c93a7ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:54.922399 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f8d56-e125-42b3-be4e-a7e9c93a7ded None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:54.922688 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f8d56-e125-42b3-be4e-a7e9c93a7ded None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:54.922999 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f8d56-e125-42b3-be4e-a7e9c93a7ded None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:54.936168 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4f8d56-e125-42b3-be4e-a7e9c93a7ded None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:01:54.944744 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-be4f8d56-e125-42b3-be4e-a7e9c93a7ded None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 02 19:01:54.944744 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:54.944744 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:54.944744 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:54.944744 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:54.944744 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:54.944744 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:54.944744 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:54.944744 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:54.944744 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:54.944744 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:54.944744 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:54.944744 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:01:54.944744 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:01:54.944744 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:01:54.944744 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:01:54.946696 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:01:54.946696 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:01:54.946696 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:01:54.946696 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:01:54.946696 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:54.946696 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:54.946696 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:54.946696 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:54.946696 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:54.946696 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:54.946696 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:54.946696 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:54.946696 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:54.946696 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:54.946696 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:54.946696 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:54.948647 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:54.948647 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:54.948647 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:54.948647 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:54.948647 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:01:54.948647 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:01:54.948647 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:01:54.948647 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:01:54.948647 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:01:54.948647 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:01:54.948647 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 02 19:01:54.948647 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:01:54.951013 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 540/1087] 10.222.0.33 () {56 vars in 1120 bytes} [Thu Jun 2 19:01:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:54.953253 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa40af6-aebf-4bc2-bf30-0188d7ccaaf0 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:55.030831 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:55.074790 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa40af6-aebf-4bc2-bf30-0188d7ccaaf0 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:55.123956 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898ab137-0b91-4b3c-8b5a-cc0ce3e0a921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:55.124727 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898ab137-0b91-4b3c-8b5a-cc0ce3e0a921 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:55.125075 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898ab137-0b91-4b3c-8b5a-cc0ce3e0a921 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:55.125412 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898ab137-0b91-4b3c-8b5a-cc0ce3e0a921 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:55.126292 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898ab137-0b91-4b3c-8b5a-cc0ce3e0a921 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:55.126687 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898ab137-0b91-4b3c-8b5a-cc0ce3e0a921 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:55.128180 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898ab137-0b91-4b3c-8b5a-cc0ce3e0a921 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:55.138348 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 548/1088] 10.222.0.33 () {56 vars in 1115 bytes} [Thu Jun 2 19:01:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:55.144765 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4f8d56-e125-42b3-be4e-a7e9c93a7ded None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:55.149446 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:55.326298 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4f8d56-e125-42b3-be4e-a7e9c93a7ded None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:55.386233 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f95206f-a097-4d49-8290-3e2795ced5c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:55.387618 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f95206f-a097-4d49-8290-3e2795ced5c1 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:55.388128 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f95206f-a097-4d49-8290-3e2795ced5c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:55.388692 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f95206f-a097-4d49-8290-3e2795ced5c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:55.409538 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f95206f-a097-4d49-8290-3e2795ced5c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:55.412357 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f95206f-a097-4d49-8290-3e2795ced5c1 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:55.445406 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-7f95206f-a097-4d49-8290-3e2795ced5c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:01:55.445406 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:55.445406 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:55.445406 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:55.445406 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:55.445406 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:55.445406 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:55.445406 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:55.445406 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:55.445406 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:55.445406 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:55.445406 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:55.445406 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:01:55.445406 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:01:55.445406 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:01:55.447026 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:01:55.447026 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:55.447026 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:55.447026 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:55.447026 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:55.447026 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:55.447026 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:55.447026 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:55.447026 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:55.447026 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:55.447026 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:55.447026 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:55.447026 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:55.447026 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:55.447026 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:55.447026 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:55.447026 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:55.454001 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:01:55.454001 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:01:55.454001 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:01:55.454001 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:01:55.454001 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 541/1089] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:01:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:55.454622 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898ab137-0b91-4b3c-8b5a-cc0ce3e0a921 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:55.561844 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898ab137-0b91-4b3c-8b5a-cc0ce3e0a921 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:55.589246 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:55.599055 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d250164-3c14-4473-ad12-3dcabf4dbd6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:55.599822 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d250164-3c14-4473-ad12-3dcabf4dbd6b None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:55.601347 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d250164-3c14-4473-ad12-3dcabf4dbd6b None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:55.601347 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d250164-3c14-4473-ad12-3dcabf4dbd6b None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:55.601347 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d250164-3c14-4473-ad12-3dcabf4dbd6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:55.601752 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d250164-3c14-4473-ad12-3dcabf4dbd6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:55.603953 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d250164-3c14-4473-ad12-3dcabf4dbd6b None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:55.616714 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 549/1090] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:01:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:55.623684 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f95206f-a097-4d49-8290-3e2795ced5c1 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:55.692744 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:55.743301 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f95206f-a097-4d49-8290-3e2795ced5c1 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:55.782993 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018fd63a-98e6-4a68-addb-52525f4fb649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:55.783879 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018fd63a-98e6-4a68-addb-52525f4fb649 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:55.784222 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018fd63a-98e6-4a68-addb-52525f4fb649 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:55.784558 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018fd63a-98e6-4a68-addb-52525f4fb649 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:55.794090 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018fd63a-98e6-4a68-addb-52525f4fb649 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:55.795704 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018fd63a-98e6-4a68-addb-52525f4fb649 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:55.811034 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-018fd63a-98e6-4a68-addb-52525f4fb649 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 02 19:01:55.811034 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:55.811034 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:55.811034 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:55.811034 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:55.811034 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:55.811034 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:55.811034 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:55.811034 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:55.811034 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:55.811034 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:55.811034 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:55.811034 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 02 19:01:55.811034 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 02 19:01:55.811034 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 02 19:01:55.811034 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 02 19:01:55.813040 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:55.813040 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:55.813040 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 02 19:01:55.813040 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:01:55.813040 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 02 19:01:55.813040 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:01:55.813040 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:55.813040 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:55.813040 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:55.813040 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:55.813040 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:55.813040 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:55.813040 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:55.813040 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:55.813040 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:55.813040 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:55.813040 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:55.813040 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:55.815771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:55.815771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:55.815771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 02 19:01:55.815771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 02 19:01:55.815771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 02 19:01:55.815771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 02 19:01:55.815771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 02 19:01:55.815771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 02 19:01:55.815771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 02 19:01:55.815771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:01:55.815771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 542/1091] 10.222.0.33 () {56 vars in 1093 bytes} [Thu Jun 2 19:01:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:01:55.820511 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d250164-3c14-4473-ad12-3dcabf4dbd6b None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:55.992760 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d250164-3c14-4473-ad12-3dcabf4dbd6b None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:56.055745 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cf040b-f9f5-4579-a05a-6b06f40220a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:56.057478 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf040b-f9f5-4579-a05a-6b06f40220a8 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:56.057991 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf040b-f9f5-4579-a05a-6b06f40220a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:56.058500 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf040b-f9f5-4579-a05a-6b06f40220a8 None admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:56.060192 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cf040b-f9f5-4579-a05a-6b06f40220a8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:56.060762 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cf040b-f9f5-4579-a05a-6b06f40220a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:56.063707 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cf040b-f9f5-4579-a05a-6b06f40220a8 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:56.100331 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 550/1092] 10.222.0.33 () {56 vars in 1133 bytes} [Thu Jun 2 19:01:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:01:56.106212 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018fd63a-98e6-4a68-addb-52525f4fb649 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:56.277727 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018fd63a-98e6-4a68-addb-52525f4fb649 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:56.293660 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:56.316009 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8eb335-16de-4533-b4c9-275531f1e492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:56.316666 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8eb335-16de-4533-b4c9-275531f1e492 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:56.317048 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8eb335-16de-4533-b4c9-275531f1e492 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:56.317424 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8eb335-16de-4533-b4c9-275531f1e492 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:56.394340 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8eb335-16de-4533-b4c9-275531f1e492 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:56.395981 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8eb335-16de-4533-b4c9-275531f1e492 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:56.414676 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7c2994-9c0d-43a5-9e16-b59960ea43a1 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:56.415156 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7c2994-9c0d-43a5-9e16-b59960ea43a1 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:56.415604 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7c2994-9c0d-43a5-9e16-b59960ea43a1 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:56.418063 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 551/1093] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:01:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 02 19:01:56.424222 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c402a912-a2d4-4b78-a9b9-23452f49076e None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:56.424566 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c402a912-a2d4-4b78-a9b9-23452f49076e None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:56.425032 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c402a912-a2d4-4b78-a9b9-23452f49076e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:56.453150 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-c402a912-a2d4-4b78-a9b9-23452f49076e None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:01:56.455284 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-c402a912-a2d4-4b78-a9b9-23452f49076e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:01:56.506475 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 543/1094] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 2 19:01:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 02 19:01:56.516332 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8eb335-16de-4533-b4c9-275531f1e492 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:56.524423 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:56.538865 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:56.553252 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 552/1095] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:01:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 02 19:01:56.582135 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c402a912-a2d4-4b78-a9b9-23452f49076e None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:56.632298 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8eb335-16de-4533-b4c9-275531f1e492 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:56.675160 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ec1582-a1b4-453a-93b7-3db951490585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:56.676280 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ec1582-a1b4-453a-93b7-3db951490585 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:56.676655 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ec1582-a1b4-453a-93b7-3db951490585 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:56.677028 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ec1582-a1b4-453a-93b7-3db951490585 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:56.689579 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ec1582-a1b4-453a-93b7-3db951490585 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:56.691396 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ec1582-a1b4-453a-93b7-3db951490585 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:56.702437 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c402a912-a2d4-4b78-a9b9-23452f49076e None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:56.712339 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-28ec1582-a1b4-453a-93b7-3db951490585 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:01:56.712339 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:56.712339 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:56.712339 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:56.712339 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:56.712339 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:56.712339 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:56.712339 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:56.712339 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:56.712339 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:56.712339 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:56.712339 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:56.712339 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:01:56.712339 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:01:56.712339 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:01:56.714920 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:01:56.714920 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:56.714920 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:56.714920 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:56.714920 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:56.714920 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:56.714920 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:56.714920 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:56.714920 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:56.714920 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:56.714920 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:56.714920 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:56.714920 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:56.714920 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:56.714920 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:56.714920 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:56.714920 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:56.716633 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:01:56.716633 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:01:56.716633 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:01:56.716633 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:01:56.716633 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 544/1096] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:01:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 02 19:01:56.721356 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ec1582-a1b4-453a-93b7-3db951490585 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:56.747538 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15986422-ec8e-4810-9170-8c393413f176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:56.748583 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15986422-ec8e-4810-9170-8c393413f176 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:56.749003 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15986422-ec8e-4810-9170-8c393413f176 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:56.749422 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15986422-ec8e-4810-9170-8c393413f176 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:56.765451 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15986422-ec8e-4810-9170-8c393413f176 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:01:56.774921 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-15986422-ec8e-4810-9170-8c393413f176 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 02 19:01:56.774921 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:56.774921 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:56.774921 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:56.774921 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:56.774921 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:56.774921 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:56.774921 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:56.774921 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:56.774921 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:56.774921 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:56.774921 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:56.774921 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:01:56.774921 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:01:56.774921 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:01:56.774921 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:01:56.777156 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:01:56.777156 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:01:56.777156 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:01:56.777156 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:01:56.777156 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:56.777156 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:56.777156 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:56.777156 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:56.777156 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:56.777156 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:56.777156 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:56.777156 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:56.777156 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:56.777156 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:56.777156 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:56.777156 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:56.778668 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:56.778668 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:56.778668 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:56.778668 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:56.778668 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:01:56.778668 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:01:56.778668 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:01:56.778668 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:01:56.778668 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:01:56.778668 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:01:56.778668 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 02 19:01:56.778668 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:01:56.778668 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 553/1097] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 2 19:01:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 02 19:01:56.784964 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15986422-ec8e-4810-9170-8c393413f176 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:56.816219 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:56.835189 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:56.864489 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ec1582-a1b4-453a-93b7-3db951490585 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:56.918686 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29e7185-bce8-43d8-b3bc-60ba059ce812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:56.919541 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29e7185-bce8-43d8-b3bc-60ba059ce812 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:56.920061 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29e7185-bce8-43d8-b3bc-60ba059ce812 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:56.920409 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29e7185-bce8-43d8-b3bc-60ba059ce812 None admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:56.921288 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29e7185-bce8-43d8-b3bc-60ba059ce812 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:56.921730 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29e7185-bce8-43d8-b3bc-60ba059ce812 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:56.923595 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29e7185-bce8-43d8-b3bc-60ba059ce812 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:56.935090 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15986422-ec8e-4810-9170-8c393413f176 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:56.938622 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 545/1098] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:01:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:01:56.945021 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29e7185-bce8-43d8-b3bc-60ba059ce812 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:56.984997 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46497148-7622-4500-ab37-34cbc25a069c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:56.986044 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46497148-7622-4500-ab37-34cbc25a069c None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:56.986481 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46497148-7622-4500-ab37-34cbc25a069c None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:56.986977 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46497148-7622-4500-ab37-34cbc25a069c None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:56.988298 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46497148-7622-4500-ab37-34cbc25a069c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:56.988818 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46497148-7622-4500-ab37-34cbc25a069c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:56.991165 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46497148-7622-4500-ab37-34cbc25a069c None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:57.005845 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 554/1099] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 2 19:01:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:01:57.013132 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46497148-7622-4500-ab37-34cbc25a069c None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:57.082450 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29e7185-bce8-43d8-b3bc-60ba059ce812 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:57.113245 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:57.126974 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82ea3f3-9240-46fd-9657-731ff92d968b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:57.128241 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82ea3f3-9240-46fd-9657-731ff92d968b None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:57.128688 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82ea3f3-9240-46fd-9657-731ff92d968b None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:57.129014 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82ea3f3-9240-46fd-9657-731ff92d968b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:57.140128 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:57.141459 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82ea3f3-9240-46fd-9657-731ff92d968b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:57.144729 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82ea3f3-9240-46fd-9657-731ff92d968b None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:57.149315 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46497148-7622-4500-ab37-34cbc25a069c None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:57.167476 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-b82ea3f3-9240-46fd-9657-731ff92d968b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:01:57.167476 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:01:57.167476 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:01:57.167476 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:01:57.167476 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:01:57.167476 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:01:57.167476 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:01:57.167476 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:01:57.167476 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:01:57.167476 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:01:57.167476 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:01:57.167476 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:01:57.167476 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 02 19:01:57.167476 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 02 19:01:57.167476 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 02 19:01:57.168499 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 02 19:01:57.168499 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:01:57.168499 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:01:57.168499 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:01:57.168499 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:01:57.168499 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:01:57.168499 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:01:57.168499 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:01:57.168499 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:01:57.168499 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:01:57.168499 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:01:57.168499 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:01:57.168499 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:01:57.168499 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:01:57.168499 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:01:57.168499 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:01:57.168499 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:01:57.169384 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 02 19:01:57.169384 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 02 19:01:57.169384 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 02 19:01:57.169384 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 02 19:01:57.169384 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 02 19:01:57.169384 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 02 19:01:57.169384 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:01:57.169384 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:01:57.169384 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 546/1100] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 2 19:01:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 02 19:01:57.175925 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82ea3f3-9240-46fd-9657-731ff92d968b None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:57.209714 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f340c34-12cb-4523-80d5-6c8323c07600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:57.210610 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f340c34-12cb-4523-80d5-6c8323c07600 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:57.210930 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f340c34-12cb-4523-80d5-6c8323c07600 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:57.211220 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f340c34-12cb-4523-80d5-6c8323c07600 None admin] PATH_INFO: `/v3/users/3b414e92fbf34dd19ff334cac4e4effe` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:57.212466 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f340c34-12cb-4523-80d5-6c8323c07600 None admin] RBAC: Authorizing `identity:get_user(user_id=3b414e92fbf34dd19ff334cac4e4effe)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:57.213992 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f340c34-12cb-4523-80d5-6c8323c07600 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:57.215771 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 555/1101] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 2 19:01:57 2022] GET /identity/v3/users/3b414e92fbf34dd19ff334cac4e4effe => generated 285 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:01:57.221622 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f340c34-12cb-4523-80d5-6c8323c07600 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:57.300107 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82ea3f3-9240-46fd-9657-731ff92d968b None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:57.332716 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f340c34-12cb-4523-80d5-6c8323c07600 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:57.342588 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b634bf95-ca36-4102-b172-04e0b16cbbce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:57.343293 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b634bf95-ca36-4102-b172-04e0b16cbbce None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:57.343582 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b634bf95-ca36-4102-b172-04e0b16cbbce None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:57.343857 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b634bf95-ca36-4102-b172-04e0b16cbbce None admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:57.344690 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b634bf95-ca36-4102-b172-04e0b16cbbce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:57.345000 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b634bf95-ca36-4102-b172-04e0b16cbbce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:57.346444 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b634bf95-ca36-4102-b172-04e0b16cbbce None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:57.362524 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 547/1102] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 2 19:01:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:01:57.368764 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b634bf95-ca36-4102-b172-04e0b16cbbce None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:57.369405 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11c29e0-7e3a-405d-ab25-28b0cdba16b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:57.370076 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c29e0-7e3a-405d-ab25-28b0cdba16b8 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:57.370364 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c29e0-7e3a-405d-ab25-28b0cdba16b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:57.370646 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c29e0-7e3a-405d-ab25-28b0cdba16b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:57.404330 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:57.437096 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:57.444339 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11c29e0-7e3a-405d-ab25-28b0cdba16b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:57.446243 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11c29e0-7e3a-405d-ab25-28b0cdba16b8 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:57.491764 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b634bf95-ca36-4102-b172-04e0b16cbbce None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:57.527968 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0553d2d6-6cbf-41c1-8b14-734806c1a6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:57.528638 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0553d2d6-6cbf-41c1-8b14-734806c1a6cd None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:57.528924 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0553d2d6-6cbf-41c1-8b14-734806c1a6cd None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:57.529198 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0553d2d6-6cbf-41c1-8b14-734806c1a6cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:57.530067 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0553d2d6-6cbf-41c1-8b14-734806c1a6cd None admin] RBAC: Adding query filter params (role.id=8d5938aa1af04179bc44ffc0e0f376d9, scope.project.id=8c04020eaebd4647804cf8653f32d9e5, user.id=70664d5cbdee47ee89c120761939cc75) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:57.530386 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0553d2d6-6cbf-41c1-8b14-734806c1a6cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:57.531868 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0553d2d6-6cbf-41c1-8b14-734806c1a6cd None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:57.538484 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 548/1103] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 2 19:01:57 2022] GET /identity/v3/role_assignments?user.id=70664d5cbdee47ee89c120761939cc75&scope.project.id=8c04020eaebd4647804cf8653f32d9e5&role.id=8d5938aa1af04179bc44ffc0e0f376d9 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:01:57.583720 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 556/1104] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 2 19:01:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 02 19:01:57.589841 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0553d2d6-6cbf-41c1-8b14-734806c1a6cd None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:57.715887 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0553d2d6-6cbf-41c1-8b14-734806c1a6cd None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:57.728074 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:57.758589 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02086601-6b09-4c47-86a6-add7a2cfeace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:57.759584 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02086601-6b09-4c47-86a6-add7a2cfeace None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:57.760486 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02086601-6b09-4c47-86a6-add7a2cfeace None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:57.760834 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02086601-6b09-4c47-86a6-add7a2cfeace None admin] PATH_INFO: `/v3/projects/f3f51e06fcfe468188953e05a231ce9d` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:57.762552 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02086601-6b09-4c47-86a6-add7a2cfeace None admin] RBAC: Authorizing `identity:get_project(project_id=f3f51e06fcfe468188953e05a231ce9d)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:57.764725 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02086601-6b09-4c47-86a6-add7a2cfeace None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:57.766718 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 549/1105] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 2 19:01:57 2022] GET /identity/v3/projects/f3f51e06fcfe468188953e05a231ce9d => generated 311 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:01:57.773570 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11c29e0-7e3a-405d-ab25-28b0cdba16b8 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:01:57.863295 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:01:57.907212 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11c29e0-7e3a-405d-ab25-28b0cdba16b8 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:01:57.943495 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d063af5d-434b-42c3-83a1-d45421cb7a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:01:57.944167 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d063af5d-434b-42c3-83a1-d45421cb7a09 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:01:57.944460 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d063af5d-434b-42c3-83a1-d45421cb7a09 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:01:57.944738 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d063af5d-434b-42c3-83a1-d45421cb7a09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:01:57.945573 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d063af5d-434b-42c3-83a1-d45421cb7a09 None admin] RBAC: Adding query filter params (role.id=f13fdd0847924014abf1e95594195e78, scope.project.id=f3f51e06fcfe468188953e05a231ce9d, user.id=3b414e92fbf34dd19ff334cac4e4effe) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:01:57.945891 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d063af5d-434b-42c3-83a1-d45421cb7a09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:01:57.947962 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d063af5d-434b-42c3-83a1-d45421cb7a09 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:01:57.954913 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 557/1106] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 2 19:01:57 2022] GET /identity/v3/role_assignments?user.id=3b414e92fbf34dd19ff334cac4e4effe&scope.project.id=f3f51e06fcfe468188953e05a231ce9d&role.id=f13fdd0847924014abf1e95594195e78 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:02:00.351860 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91a26c-4249-4118-8b13-c966ba2c7e98 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:00.352478 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91a26c-4249-4118-8b13-c966ba2c7e98 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:00.352478 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91a26c-4249-4118-8b13-c966ba2c7e98 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:00.354430 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 550/1107] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:02:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 02 19:02:00.360262 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcfbdc9-40f2-447b-b763-a316957e9d91 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:00.360755 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcfbdc9-40f2-447b-b763-a316957e9d91 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:00.360881 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcfbdc9-40f2-447b-b763-a316957e9d91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:00.378581 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-edcfbdc9-40f2-447b-b763-a316957e9d91 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:02:00.379768 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-edcfbdc9-40f2-447b-b763-a316957e9d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:02:00.450666 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 558/1108] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:02:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 02 19:02:00.476106 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc91a26c-4249-4118-8b13-c966ba2c7e98 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:02:00.588619 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc91a26c-4249-4118-8b13-c966ba2c7e98 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:02:00.624135 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c887c79a-cdd9-4b50-958d-c17823104aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:02:00.624987 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c887c79a-cdd9-4b50-958d-c17823104aec None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:00.625331 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c887c79a-cdd9-4b50-958d-c17823104aec None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:00.625654 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c887c79a-cdd9-4b50-958d-c17823104aec None admin] PATH_INFO: `/v3/domains/default` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:00.626981 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c887c79a-cdd9-4b50-958d-c17823104aec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:02:00.628496 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c887c79a-cdd9-4b50-958d-c17823104aec None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:02:00.630250 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 551/1109] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:02:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:02:00.637141 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcfbdc9-40f2-447b-b763-a316957e9d91 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:02:00.699928 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:02:00.741032 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcfbdc9-40f2-447b-b763-a316957e9d91 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:02:00.778856 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e993f35-7eeb-41dc-8197-38b0ba762707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:02:00.779574 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e993f35-7eeb-41dc-8197-38b0ba762707 None admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:00.779862 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e993f35-7eeb-41dc-8197-38b0ba762707 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:00.780141 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e993f35-7eeb-41dc-8197-38b0ba762707 None admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:00.780914 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e993f35-7eeb-41dc-8197-38b0ba762707 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:02:00.782338 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e993f35-7eeb-41dc-8197-38b0ba762707 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:02:00.879569 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 559/1110] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 2 19:02:00 2022] POST /identity/v3/users => generated 286 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:02:03.272296 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df8ac7-1132-4f79-bc9f-d58c22078bfa None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:03.272900 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df8ac7-1132-4f79-bc9f-d58c22078bfa None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:03.272900 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df8ac7-1132-4f79-bc9f-d58c22078bfa None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:03.274461 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 552/1111] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:02:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 02 19:02:03.281194 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755e1a89-d059-4632-b3b6-f2fba8efb48c None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:03.281710 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755e1a89-d059-4632-b3b6-f2fba8efb48c None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:03.281844 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755e1a89-d059-4632-b3b6-f2fba8efb48c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:03.301075 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-755e1a89-d059-4632-b3b6-f2fba8efb48c None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:02:03.302425 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-755e1a89-d059-4632-b3b6-f2fba8efb48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:02:03.380732 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 560/1112] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:02:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 02 19:02:03.406575 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4df8ac7-1132-4f79-bc9f-d58c22078bfa None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:02:03.462432 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:02:03.526043 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4df8ac7-1132-4f79-bc9f-d58c22078bfa None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:02:03.563622 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecffe35d-b8dc-4632-af06-6ac69e01a59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:02:03.564502 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecffe35d-b8dc-4632-af06-6ac69e01a59f None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:03.564784 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecffe35d-b8dc-4632-af06-6ac69e01a59f None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:03.565064 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecffe35d-b8dc-4632-af06-6ac69e01a59f None admin] PATH_INFO: `/v3/roles/2e0c054b646e4ed2a0e7e5962f09a50e` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:03.566606 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecffe35d-b8dc-4632-af06-6ac69e01a59f None admin] RBAC: Authorizing `identity:get_role(role_id=2e0c054b646e4ed2a0e7e5962f09a50e)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:02:03.568052 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecffe35d-b8dc-4632-af06-6ac69e01a59f None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:02:03.569445 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 553/1113] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 2 19:02:03 2022] GET /identity/v3/roles/2e0c054b646e4ed2a0e7e5962f09a50e => generated 223 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:02:03.575489 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755e1a89-d059-4632-b3b6-f2fba8efb48c None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:02:03.624874 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:02:03.684622 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755e1a89-d059-4632-b3b6-f2fba8efb48c None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:02:03.723876 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23facd43-2d27-44bb-b057-714159cf72c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:02:03.724833 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23facd43-2d27-44bb-b057-714159cf72c3 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:03.725166 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23facd43-2d27-44bb-b057-714159cf72c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:03.725494 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23facd43-2d27-44bb-b057-714159cf72c3 None admin] PATH_INFO: `/v3/users/f96dd950b76a45e7b64854268826af26` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:03.741239 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23facd43-2d27-44bb-b057-714159cf72c3 None admin] RBAC: Authorizing `identity:get_user(user_id=f96dd950b76a45e7b64854268826af26)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:02:03.742863 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23facd43-2d27-44bb-b057-714159cf72c3 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:02:03.745347 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 561/1114] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 2 19:02:03 2022] GET /identity/v3/users/f96dd950b76a45e7b64854268826af26 => generated 286 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:02:03.751661 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecffe35d-b8dc-4632-af06-6ac69e01a59f None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:02:03.879870 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecffe35d-b8dc-4632-af06-6ac69e01a59f None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:02:03.896434 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:02:03.919085 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599339ba-0829-4a55-848d-9f517d0041ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:02:03.919753 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599339ba-0829-4a55-848d-9f517d0041ca None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:03.920049 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599339ba-0829-4a55-848d-9f517d0041ca None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:03.920410 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599339ba-0829-4a55-848d-9f517d0041ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:03.999699 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599339ba-0829-4a55-848d-9f517d0041ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:02:04.001190 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599339ba-0829-4a55-848d-9f517d0041ca None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:02:04.111461 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 554/1115] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 2 19:02:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 02 19:02:04.117860 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23facd43-2d27-44bb-b057-714159cf72c3 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:02:04.226502 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23facd43-2d27-44bb-b057-714159cf72c3 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:02:04.228966 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:02:04.265392 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1ce519-59d1-4b50-8ec4-c2b1a0fcabe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:02:04.266302 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1ce519-59d1-4b50-8ec4-c2b1a0fcabe6 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:04.266634 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1ce519-59d1-4b50-8ec4-c2b1a0fcabe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:04.267021 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1ce519-59d1-4b50-8ec4-c2b1a0fcabe6 None admin] PATH_INFO: `/v3/projects/f3f51e06fcfe468188953e05a231ce9d` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:04.268390 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1ce519-59d1-4b50-8ec4-c2b1a0fcabe6 None admin] RBAC: Authorizing `identity:get_project(project_id=f3f51e06fcfe468188953e05a231ce9d)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:02:04.269928 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1ce519-59d1-4b50-8ec4-c2b1a0fcabe6 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:02:04.271834 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 562/1116] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 2 19:02:04 2022] GET /identity/v3/projects/f3f51e06fcfe468188953e05a231ce9d => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:02:04.279484 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599339ba-0829-4a55-848d-9f517d0041ca None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:02:04.297102 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:02:04.395116 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599339ba-0829-4a55-848d-9f517d0041ca None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:02:04.436712 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8f8cf8-3193-40fa-9c63-6ab3f7be5db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:02:04.437382 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8f8cf8-3193-40fa-9c63-6ab3f7be5db0 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:04.437664 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8f8cf8-3193-40fa-9c63-6ab3f7be5db0 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:04.437941 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8f8cf8-3193-40fa-9c63-6ab3f7be5db0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:04.438903 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8f8cf8-3193-40fa-9c63-6ab3f7be5db0 None admin] RBAC: Adding query filter params (role.id=2e0c054b646e4ed2a0e7e5962f09a50e, scope.project.id=f3f51e06fcfe468188953e05a231ce9d, user.id=f96dd950b76a45e7b64854268826af26) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:02:04.439256 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8f8cf8-3193-40fa-9c63-6ab3f7be5db0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:02:04.440707 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8f8cf8-3193-40fa-9c63-6ab3f7be5db0 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:02:04.447742 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 555/1117] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 2 19:02:04 2022] GET /identity/v3/role_assignments?user.id=f96dd950b76a45e7b64854268826af26&scope.project.id=f3f51e06fcfe468188953e05a231ce9d&role.id=2e0c054b646e4ed2a0e7e5962f09a50e => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:02:06.768847 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f548343c-5771-4b55-844b-6b697cf47d68 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:06.769398 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f548343c-5771-4b55-844b-6b697cf47d68 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:06.769591 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f548343c-5771-4b55-844b-6b697cf47d68 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:06.771236 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 563/1118] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:02:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:02:06.777705 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2889e5-0a1f-4c2c-a0a7-caddc6d981b7 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:06.778222 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2889e5-0a1f-4c2c-a0a7-caddc6d981b7 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:06.778346 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2889e5-0a1f-4c2c-a0a7-caddc6d981b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:06.796938 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-3a2889e5-0a1f-4c2c-a0a7-caddc6d981b7 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:02:06.798220 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-3a2889e5-0a1f-4c2c-a0a7-caddc6d981b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:02:06.873165 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 556/1119] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:02:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 02 19:02:06.897173 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f548343c-5771-4b55-844b-6b697cf47d68 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:02:07.001691 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f548343c-5771-4b55-844b-6b697cf47d68 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:02:07.036433 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cb6776-8c77-4eb8-baa1-3d629372a9ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:02:07.037208 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb6776-8c77-4eb8-baa1-3d629372a9ec None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:07.037487 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb6776-8c77-4eb8-baa1-3d629372a9ec None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:07.037760 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb6776-8c77-4eb8-baa1-3d629372a9ec None admin] PATH_INFO: `/v3/roles/2e0c054b646e4ed2a0e7e5962f09a50e` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:07.039269 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cb6776-8c77-4eb8-baa1-3d629372a9ec None admin] RBAC: Authorizing `identity:get_role(role_id=2e0c054b646e4ed2a0e7e5962f09a50e)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:02:07.040645 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cb6776-8c77-4eb8-baa1-3d629372a9ec None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:02:07.042116 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 564/1120] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 2 19:02:06 2022] GET /identity/v3/roles/2e0c054b646e4ed2a0e7e5962f09a50e => generated 223 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:02:07.048343 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2889e5-0a1f-4c2c-a0a7-caddc6d981b7 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:02:07.060403 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:02:07.163540 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2889e5-0a1f-4c2c-a0a7-caddc6d981b7 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:02:07.202049 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046a2029-1b96-468c-a579-74049940c8ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:02:07.203159 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046a2029-1b96-468c-a579-74049940c8ad None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:07.203613 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046a2029-1b96-468c-a579-74049940c8ad None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:07.204080 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046a2029-1b96-468c-a579-74049940c8ad None admin] PATH_INFO: `/v3/users/f96dd950b76a45e7b64854268826af26` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:07.205409 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046a2029-1b96-468c-a579-74049940c8ad None admin] RBAC: Authorizing `identity:get_user(user_id=f96dd950b76a45e7b64854268826af26)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:02:07.206956 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046a2029-1b96-468c-a579-74049940c8ad None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:02:07.208670 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 557/1121] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 2 19:02:07 2022] GET /identity/v3/users/f96dd950b76a45e7b64854268826af26 => generated 286 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:02:07.214770 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cb6776-8c77-4eb8-baa1-3d629372a9ec None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:02:07.260751 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:02:07.328625 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cb6776-8c77-4eb8-baa1-3d629372a9ec None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:02:07.367465 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38da65b-a6f8-495e-8897-5773077163b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:02:07.368375 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38da65b-a6f8-495e-8897-5773077163b9 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:07.368663 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38da65b-a6f8-495e-8897-5773077163b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:07.369018 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38da65b-a6f8-495e-8897-5773077163b9 None admin] PATH_INFO: `/v3/projects/f3f51e06fcfe468188953e05a231ce9d` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:07.370425 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38da65b-a6f8-495e-8897-5773077163b9 None admin] RBAC: Authorizing `identity:get_project(project_id=f3f51e06fcfe468188953e05a231ce9d)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:02:07.372001 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38da65b-a6f8-495e-8897-5773077163b9 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:02:07.373886 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 565/1122] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 2 19:02:07 2022] GET /identity/v3/projects/f3f51e06fcfe468188953e05a231ce9d => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:02:07.380331 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046a2029-1b96-468c-a579-74049940c8ad None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:02:07.466207 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:02:07.489816 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046a2029-1b96-468c-a579-74049940c8ad None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:02:07.528250 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc13c68-b205-4144-86de-e2d696fd0e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:02:07.529148 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc13c68-b205-4144-86de-e2d696fd0e22 None admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:07.529432 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc13c68-b205-4144-86de-e2d696fd0e22 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:07.529709 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc13c68-b205-4144-86de-e2d696fd0e22 None admin] PATH_INFO: `/v3/projects/f3f51e06fcfe468188953e05a231ce9d/users/f96dd950b76a45e7b64854268826af26/roles/2e0c054b646e4ed2a0e7e5962f09a50e` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:07.531947 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc13c68-b205-4144-86de-e2d696fd0e22 None admin] RBAC: Authorizing `identity:create_grant(project_id=f3f51e06fcfe468188953e05a231ce9d, user_id=f96dd950b76a45e7b64854268826af26, role_id=2e0c054b646e4ed2a0e7e5962f09a50e)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:02:07.533391 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc13c68-b205-4144-86de-e2d696fd0e22 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:02:07.656718 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 558/1123] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 2 19:02:07 2022] PUT /identity/v3/projects/f3f51e06fcfe468188953e05a231ce9d/users/f96dd950b76a45e7b64854268826af26/roles/2e0c054b646e4ed2a0e7e5962f09a50e => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:02:09.994307 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46903be-8982-4cda-8605-9e12705fd7c7 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:09.994911 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46903be-8982-4cda-8605-9e12705fd7c7 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:09.995358 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46903be-8982-4cda-8605-9e12705fd7c7 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:09.998180 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 566/1124] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:02:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:02:10.004500 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69d0303-aa8b-41b8-be50-3fa5d0689425 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:10.005058 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69d0303-aa8b-41b8-be50-3fa5d0689425 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:10.005058 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69d0303-aa8b-41b8-be50-3fa5d0689425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:10.024350 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-f69d0303-aa8b-41b8-be50-3fa5d0689425 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:02:10.026221 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-f69d0303-aa8b-41b8-be50-3fa5d0689425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:02:10.105209 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 559/1125] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:02:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 02 19:02:10.129959 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46903be-8982-4cda-8605-9e12705fd7c7 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:02:10.263665 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46903be-8982-4cda-8605-9e12705fd7c7 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:02:10.272119 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:02:10.304705 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd94b8a6-2f88-41d2-abf7-ada881a1b256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:02:10.305596 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd94b8a6-2f88-41d2-abf7-ada881a1b256 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:10.305887 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd94b8a6-2f88-41d2-abf7-ada881a1b256 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:10.306169 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd94b8a6-2f88-41d2-abf7-ada881a1b256 None admin] PATH_INFO: `/v3/roles/2e0c054b646e4ed2a0e7e5962f09a50e` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:10.308448 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd94b8a6-2f88-41d2-abf7-ada881a1b256 None admin] RBAC: Authorizing `identity:get_role(role_id=2e0c054b646e4ed2a0e7e5962f09a50e)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:02:10.309892 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd94b8a6-2f88-41d2-abf7-ada881a1b256 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:02:10.311452 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 567/1126] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 2 19:02:10 2022] GET /identity/v3/roles/2e0c054b646e4ed2a0e7e5962f09a50e => generated 223 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:02:10.319899 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69d0303-aa8b-41b8-be50-3fa5d0689425 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:02:10.397943 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:02:10.437381 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69d0303-aa8b-41b8-be50-3fa5d0689425 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:02:10.476517 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac163b7f-14f9-4a2b-a395-a391ad4327ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:02:10.477479 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac163b7f-14f9-4a2b-a395-a391ad4327ad None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:10.477841 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac163b7f-14f9-4a2b-a395-a391ad4327ad None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:10.478248 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac163b7f-14f9-4a2b-a395-a391ad4327ad None admin] PATH_INFO: `/v3/users/f96dd950b76a45e7b64854268826af26` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:10.479882 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac163b7f-14f9-4a2b-a395-a391ad4327ad None admin] RBAC: Authorizing `identity:get_user(user_id=f96dd950b76a45e7b64854268826af26)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:02:10.481961 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac163b7f-14f9-4a2b-a395-a391ad4327ad None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:02:10.484478 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 560/1127] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 2 19:02:10 2022] GET /identity/v3/users/f96dd950b76a45e7b64854268826af26 => generated 286 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:02:10.490800 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd94b8a6-2f88-41d2-abf7-ada881a1b256 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:02:10.596516 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd94b8a6-2f88-41d2-abf7-ada881a1b256 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:02:10.632817 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c0a7da-4886-4b34-a2f5-9bc15b59250d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:02:10.633473 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c0a7da-4886-4b34-a2f5-9bc15b59250d None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:10.633754 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c0a7da-4886-4b34-a2f5-9bc15b59250d None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:10.634026 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c0a7da-4886-4b34-a2f5-9bc15b59250d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:10.677670 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:02:10.704339 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c0a7da-4886-4b34-a2f5-9bc15b59250d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:02:10.705807 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c0a7da-4886-4b34-a2f5-9bc15b59250d None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:02:10.818331 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 568/1128] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 2 19:02:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 02 19:02:10.827592 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac163b7f-14f9-4a2b-a395-a391ad4327ad None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:02:10.948990 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac163b7f-14f9-4a2b-a395-a391ad4327ad None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:02:10.989914 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ca4589-d96c-430e-a778-3483e32e4fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:02:10.990976 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ca4589-d96c-430e-a778-3483e32e4fc8 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:10.991350 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ca4589-d96c-430e-a778-3483e32e4fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:10.991731 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ca4589-d96c-430e-a778-3483e32e4fc8 None admin] PATH_INFO: `/v3/projects/f3f51e06fcfe468188953e05a231ce9d` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:10.993094 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ca4589-d96c-430e-a778-3483e32e4fc8 None admin] RBAC: Authorizing `identity:get_project(project_id=f3f51e06fcfe468188953e05a231ce9d)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:02:10.994680 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ca4589-d96c-430e-a778-3483e32e4fc8 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:02:10.996440 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 561/1129] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 2 19:02:10 2022] GET /identity/v3/projects/f3f51e06fcfe468188953e05a231ce9d => generated 311 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:02:11.006087 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c0a7da-4886-4b34-a2f5-9bc15b59250d None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:02:11.087592 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:02:11.116162 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c0a7da-4886-4b34-a2f5-9bc15b59250d None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:02:11.153914 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ab9fe9-d516-4333-874d-6a76ae2a1baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:02:11.154640 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ab9fe9-d516-4333-874d-6a76ae2a1baf None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:11.155034 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ab9fe9-d516-4333-874d-6a76ae2a1baf None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:11.155362 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ab9fe9-d516-4333-874d-6a76ae2a1baf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:11.156255 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ab9fe9-d516-4333-874d-6a76ae2a1baf None admin] RBAC: Adding query filter params (role.id=2e0c054b646e4ed2a0e7e5962f09a50e, scope.project.id=f3f51e06fcfe468188953e05a231ce9d, user.id=f96dd950b76a45e7b64854268826af26) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:02:11.156621 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ab9fe9-d516-4333-874d-6a76ae2a1baf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:02:11.158130 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ab9fe9-d516-4333-874d-6a76ae2a1baf None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:02:11.164932 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 569/1130] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 2 19:02:11 2022] GET /identity/v3/role_assignments?user.id=f96dd950b76a45e7b64854268826af26&scope.project.id=f3f51e06fcfe468188953e05a231ce9d&role.id=2e0c054b646e4ed2a0e7e5962f09a50e => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:02:13.522779 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f611b81-9f0d-4915-92c4-ab68da5d6986 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:13.523336 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f611b81-9f0d-4915-92c4-ab68da5d6986 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:13.523454 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f611b81-9f0d-4915-92c4-ab68da5d6986 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:13.524917 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 562/1131] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:02:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 02 19:02:13.531544 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2744ab-f23d-4725-855b-08fa50734d0c None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:13.532090 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2744ab-f23d-4725-855b-08fa50734d0c None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:13.532090 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2744ab-f23d-4725-855b-08fa50734d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:13.550024 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-8a2744ab-f23d-4725-855b-08fa50734d0c None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:02:13.551302 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-8a2744ab-f23d-4725-855b-08fa50734d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:02:13.624382 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 570/1132] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:02:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 02 19:02:13.649381 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f611b81-9f0d-4915-92c4-ab68da5d6986 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:02:13.653290 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:02:13.766968 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f611b81-9f0d-4915-92c4-ab68da5d6986 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:02:13.820912 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaa6b4b-6d1e-43a0-a416-316e8e924578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:02:13.822194 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa6b4b-6d1e-43a0-a416-316e8e924578 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:13.822676 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa6b4b-6d1e-43a0-a416-316e8e924578 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:13.823153 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa6b4b-6d1e-43a0-a416-316e8e924578 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:13.825254 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaa6b4b-6d1e-43a0-a416-316e8e924578 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:02:13.828050 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaa6b4b-6d1e-43a0-a416-316e8e924578 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:02:13.831469 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 563/1133] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:02:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:02:13.837516 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2744ab-f23d-4725-855b-08fa50734d0c None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:02:13.911484 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:02:13.947297 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2744ab-f23d-4725-855b-08fa50734d0c None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:02:13.984207 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a9fbec-7e15-4a40-86ce-46f873a9f154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:02:13.984856 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a9fbec-7e15-4a40-86ce-46f873a9f154 None admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:13.985142 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a9fbec-7e15-4a40-86ce-46f873a9f154 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:13.985421 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a9fbec-7e15-4a40-86ce-46f873a9f154 None admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:13.986146 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a9fbec-7e15-4a40-86ce-46f873a9f154 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:02:13.987616 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a9fbec-7e15-4a40-86ce-46f873a9f154 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:02:14.081840 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 571/1134] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Jun 2 19:02:13 2022] POST /identity/v3/projects => generated 311 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:02:16.539890 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b301689-1d5f-4347-9474-cac410fd707f None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:16.540487 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b301689-1d5f-4347-9474-cac410fd707f None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:16.540487 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b301689-1d5f-4347-9474-cac410fd707f None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:16.541975 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 564/1135] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:02:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 02 19:02:16.548573 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22326e5a-3a1f-4a88-9329-618933262d8a None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:16.549331 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22326e5a-3a1f-4a88-9329-618933262d8a None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:16.549331 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22326e5a-3a1f-4a88-9329-618933262d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:16.567298 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-22326e5a-3a1f-4a88-9329-618933262d8a None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:02:16.568534 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-22326e5a-3a1f-4a88-9329-618933262d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:02:16.642442 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 572/1136] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:02:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 02 19:02:16.669526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b301689-1d5f-4347-9474-cac410fd707f None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:02:16.750296 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:02:16.783950 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b301689-1d5f-4347-9474-cac410fd707f None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:02:16.821717 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085cdcf8-3e9e-4f84-b266-fbfe778acc41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:02:16.822444 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085cdcf8-3e9e-4f84-b266-fbfe778acc41 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:16.823329 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085cdcf8-3e9e-4f84-b266-fbfe778acc41 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:16.823612 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085cdcf8-3e9e-4f84-b266-fbfe778acc41 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:16.824869 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085cdcf8-3e9e-4f84-b266-fbfe778acc41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:02:16.826344 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085cdcf8-3e9e-4f84-b266-fbfe778acc41 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:02:16.828073 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 565/1137] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:02:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:02:16.835201 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22326e5a-3a1f-4a88-9329-618933262d8a None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:02:16.921524 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:02:17.007997 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22326e5a-3a1f-4a88-9329-618933262d8a None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:02:17.068172 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8266df-1c45-493b-bd25-4c71f2c7f0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:02:17.069306 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8266df-1c45-493b-bd25-4c71f2c7f0fc None admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:17.069803 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8266df-1c45-493b-bd25-4c71f2c7f0fc None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:17.070288 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8266df-1c45-493b-bd25-4c71f2c7f0fc None admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:17.071605 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8266df-1c45-493b-bd25-4c71f2c7f0fc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:02:17.074815 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8266df-1c45-493b-bd25-4c71f2c7f0fc None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:02:17.195481 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 573/1138] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 2 19:02:16 2022] POST /identity/v3/users => generated 286 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:02:19.594537 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd0a52c-2cb3-4172-99cd-f36d051a8a43 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:19.595143 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd0a52c-2cb3-4172-99cd-f36d051a8a43 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:19.595316 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd0a52c-2cb3-4172-99cd-f36d051a8a43 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:19.596867 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 566/1139] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:02:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 02 19:02:19.602547 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a35cdd3-0d9d-4893-bb89-2ef3bb79cfd9 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:19.603104 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a35cdd3-0d9d-4893-bb89-2ef3bb79cfd9 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:19.603229 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a35cdd3-0d9d-4893-bb89-2ef3bb79cfd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:19.621109 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-5a35cdd3-0d9d-4893-bb89-2ef3bb79cfd9 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:02:19.622351 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-5a35cdd3-0d9d-4893-bb89-2ef3bb79cfd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:02:19.700036 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 574/1140] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:02:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 02 19:02:19.724614 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd0a52c-2cb3-4172-99cd-f36d051a8a43 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:02:19.872730 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd0a52c-2cb3-4172-99cd-f36d051a8a43 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:02:19.910211 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97cc136-07e4-42c7-b3c5-dfd0672d9031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:02:19.911049 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97cc136-07e4-42c7-b3c5-dfd0672d9031 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:19.911336 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97cc136-07e4-42c7-b3c5-dfd0672d9031 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:19.911633 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97cc136-07e4-42c7-b3c5-dfd0672d9031 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:19.919867 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:02:19.923999 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97cc136-07e4-42c7-b3c5-dfd0672d9031 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:02:19.931823 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-b97cc136-07e4-42c7-b3c5-dfd0672d9031 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 02 19:02:19.931823 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:02:19.931823 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:02:19.931823 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:02:19.931823 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:02:19.931823 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:02:19.931823 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:02:19.931823 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:02:19.931823 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:02:19.931823 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:02:19.931823 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:02:19.931823 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:02:19.931823 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:02:19.931823 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:02:19.931823 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:02:19.931823 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:02:19.934691 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:02:19.934691 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:02:19.934691 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:02:19.934691 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:02:19.934691 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:02:19.934691 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:02:19.934691 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:02:19.934691 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:02:19.934691 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:02:19.934691 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:02:19.934691 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:02:19.934691 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:02:19.934691 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:02:19.934691 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:02:19.934691 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:02:19.934691 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:02:19.936259 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:02:19.936259 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:02:19.936259 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:02:19.936259 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:02:19.936259 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:02:19.936259 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:02:19.936259 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:02:19.936259 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:02:19.936259 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:02:19.936259 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:02:19.936259 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 02 19:02:19.936259 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:02:19.936259 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 567/1141] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 2 19:02:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 02 19:02:19.941204 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a35cdd3-0d9d-4893-bb89-2ef3bb79cfd9 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:02:19.981944 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:02:20.056479 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a35cdd3-0d9d-4893-bb89-2ef3bb79cfd9 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:02:20.095777 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0c3fc3-4632-4055-886f-cd82f0a294a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:02:20.096456 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0c3fc3-4632-4055-886f-cd82f0a294a6 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:20.096743 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0c3fc3-4632-4055-886f-cd82f0a294a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:20.097020 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0c3fc3-4632-4055-886f-cd82f0a294a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:20.097826 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0c3fc3-4632-4055-886f-cd82f0a294a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:02:20.098136 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0c3fc3-4632-4055-886f-cd82f0a294a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:02:20.099625 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0c3fc3-4632-4055-886f-cd82f0a294a6 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:02:20.109598 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 575/1142] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 2 19:02:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:02:20.116359 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97cc136-07e4-42c7-b3c5-dfd0672d9031 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:02:20.293634 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97cc136-07e4-42c7-b3c5-dfd0672d9031 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:02:20.331644 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d28f71e-847b-42b9-bdb7-79d84c321648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:02:20.332609 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d28f71e-847b-42b9-bdb7-79d84c321648 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:20.332944 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d28f71e-847b-42b9-bdb7-79d84c321648 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:20.333273 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d28f71e-847b-42b9-bdb7-79d84c321648 None admin] PATH_INFO: `/v3/users/19a8f99f6f27466aa99a6a23e3952c91` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:20.349669 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d28f71e-847b-42b9-bdb7-79d84c321648 None admin] RBAC: Authorizing `identity:get_user(user_id=19a8f99f6f27466aa99a6a23e3952c91)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:02:20.351272 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d28f71e-847b-42b9-bdb7-79d84c321648 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:02:20.353137 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 568/1143] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 2 19:02:20 2022] GET /identity/v3/users/19a8f99f6f27466aa99a6a23e3952c91 => generated 286 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:02:20.360338 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0c3fc3-4632-4055-886f-cd82f0a294a6 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:02:20.473391 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:02:20.477956 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0c3fc3-4632-4055-886f-cd82f0a294a6 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:02:20.519312 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ad461d-5818-4d41-b870-1cc72b23f696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:02:20.519989 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad461d-5818-4d41-b870-1cc72b23f696 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:20.520277 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad461d-5818-4d41-b870-1cc72b23f696 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:20.520560 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad461d-5818-4d41-b870-1cc72b23f696 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:20.601309 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ad461d-5818-4d41-b870-1cc72b23f696 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:02:20.602732 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ad461d-5818-4d41-b870-1cc72b23f696 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:02:20.732987 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:02:20.740132 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 576/1144] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 2 19:02:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 02 19:02:20.747075 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d28f71e-847b-42b9-bdb7-79d84c321648 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:02:20.811881 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:02:20.862649 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d28f71e-847b-42b9-bdb7-79d84c321648 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:02:20.901925 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baac6a6b-abd1-40ff-9b46-d86403fef21e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:02:20.902850 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac6a6b-abd1-40ff-9b46-d86403fef21e None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:20.903138 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac6a6b-abd1-40ff-9b46-d86403fef21e None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:20.903417 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac6a6b-abd1-40ff-9b46-d86403fef21e None admin] PATH_INFO: `/v3/projects/0729fa47cb4b41098219d643a030381c` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:20.904774 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baac6a6b-abd1-40ff-9b46-d86403fef21e None admin] RBAC: Authorizing `identity:get_project(project_id=0729fa47cb4b41098219d643a030381c)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:02:20.906260 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baac6a6b-abd1-40ff-9b46-d86403fef21e None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:02:20.908160 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 569/1145] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 2 19:02:20 2022] GET /identity/v3/projects/0729fa47cb4b41098219d643a030381c => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:02:20.915069 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ad461d-5818-4d41-b870-1cc72b23f696 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:02:21.026544 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ad461d-5818-4d41-b870-1cc72b23f696 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:02:21.073934 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338674d4-0d62-4fad-a954-a4e00a3f7f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:02:21.075108 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338674d4-0d62-4fad-a954-a4e00a3f7f6e None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:21.075610 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338674d4-0d62-4fad-a954-a4e00a3f7f6e None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:21.076114 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338674d4-0d62-4fad-a954-a4e00a3f7f6e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:21.077771 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338674d4-0d62-4fad-a954-a4e00a3f7f6e None admin] RBAC: Adding query filter params (role.id=f13fdd0847924014abf1e95594195e78, scope.project.id=0729fa47cb4b41098219d643a030381c, user.id=19a8f99f6f27466aa99a6a23e3952c91) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:02:21.078327 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338674d4-0d62-4fad-a954-a4e00a3f7f6e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:02:21.081184 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338674d4-0d62-4fad-a954-a4e00a3f7f6e None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:02:21.092663 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 577/1146] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 2 19:02:20 2022] GET /identity/v3/role_assignments?user.id=19a8f99f6f27466aa99a6a23e3952c91&scope.project.id=0729fa47cb4b41098219d643a030381c&role.id=f13fdd0847924014abf1e95594195e78 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:02:23.446601 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098cd17d-6f05-4ca1-978c-83cb0ab6a0fa None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:23.447189 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098cd17d-6f05-4ca1-978c-83cb0ab6a0fa None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:23.447189 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098cd17d-6f05-4ca1-978c-83cb0ab6a0fa None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:23.448789 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 570/1147] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:02:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:02:23.454933 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550ee8b-a863-4117-90a8-92d029af3549 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:23.455455 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550ee8b-a863-4117-90a8-92d029af3549 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:23.455548 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550ee8b-a863-4117-90a8-92d029af3549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:23.473917 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-0550ee8b-a863-4117-90a8-92d029af3549 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:02:23.475207 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-0550ee8b-a863-4117-90a8-92d029af3549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:02:23.539323 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:02:23.550930 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 578/1148] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:02:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 02 19:02:23.574612 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098cd17d-6f05-4ca1-978c-83cb0ab6a0fa None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:02:23.682461 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098cd17d-6f05-4ca1-978c-83cb0ab6a0fa None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:02:23.710858 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:02:23.723599 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea1ce4d-f5a9-4156-b38f-74964bd8a13c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:02:23.724429 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1ce4d-f5a9-4156-b38f-74964bd8a13c None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:23.724714 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1ce4d-f5a9-4156-b38f-74964bd8a13c None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:23.724993 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1ce4d-f5a9-4156-b38f-74964bd8a13c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:23.737916 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea1ce4d-f5a9-4156-b38f-74964bd8a13c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:02:23.746900 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-eea1ce4d-f5a9-4156-b38f-74964bd8a13c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 02 19:02:23.746900 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:02:23.746900 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:02:23.746900 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:02:23.746900 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:02:23.746900 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:02:23.746900 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:02:23.746900 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:02:23.746900 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:02:23.746900 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:02:23.746900 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:02:23.746900 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:02:23.746900 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:02:23.746900 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:02:23.746900 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:02:23.746900 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:02:23.748732 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:02:23.748732 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:02:23.748732 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:02:23.748732 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:02:23.748732 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:02:23.748732 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:02:23.748732 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:02:23.748732 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:02:23.748732 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:02:23.748732 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:02:23.748732 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:02:23.748732 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:02:23.748732 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:02:23.748732 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:02:23.748732 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:02:23.748732 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:02:23.750460 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:02:23.750460 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:02:23.750460 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:02:23.750460 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:02:23.750460 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:02:23.750460 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:02:23.750460 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:02:23.750460 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:02:23.750460 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:02:23.750460 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:02:23.750460 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 02 19:02:23.750460 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:02:23.750460 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 571/1149] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 2 19:02:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 02 19:02:23.757153 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0550ee8b-a863-4117-90a8-92d029af3549 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:02:23.864590 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0550ee8b-a863-4117-90a8-92d029af3549 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:02:23.901676 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eb04f1-559d-403a-ad99-1804e410f2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:02:23.902350 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eb04f1-559d-403a-ad99-1804e410f2a1 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:23.902633 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eb04f1-559d-403a-ad99-1804e410f2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:23.902940 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eb04f1-559d-403a-ad99-1804e410f2a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:23.903701 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85eb04f1-559d-403a-ad99-1804e410f2a1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:02:23.904010 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85eb04f1-559d-403a-ad99-1804e410f2a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:02:23.905398 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85eb04f1-559d-403a-ad99-1804e410f2a1 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:02:23.914434 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 579/1150] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 2 19:02:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:02:23.921253 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea1ce4d-f5a9-4156-b38f-74964bd8a13c None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:02:24.030806 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea1ce4d-f5a9-4156-b38f-74964bd8a13c None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:02:24.066738 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0212097e-78bf-4502-8e9c-adc227046759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:02:24.067656 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0212097e-78bf-4502-8e9c-adc227046759 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:24.067943 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0212097e-78bf-4502-8e9c-adc227046759 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:24.068223 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0212097e-78bf-4502-8e9c-adc227046759 None admin] PATH_INFO: `/v3/users/19a8f99f6f27466aa99a6a23e3952c91` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:24.069479 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0212097e-78bf-4502-8e9c-adc227046759 None admin] RBAC: Authorizing `identity:get_user(user_id=19a8f99f6f27466aa99a6a23e3952c91)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:02:24.070956 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0212097e-78bf-4502-8e9c-adc227046759 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:02:24.072641 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 572/1151] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 2 19:02:23 2022] GET /identity/v3/users/19a8f99f6f27466aa99a6a23e3952c91 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:02:24.079655 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eb04f1-559d-403a-ad99-1804e410f2a1 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:02:24.146014 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:02:24.194027 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eb04f1-559d-403a-ad99-1804e410f2a1 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:02:24.243193 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a54d63-23c0-4bb5-a671-049bba8b105d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:02:24.245182 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a54d63-23c0-4bb5-a671-049bba8b105d None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:24.245703 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a54d63-23c0-4bb5-a671-049bba8b105d None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:24.246212 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a54d63-23c0-4bb5-a671-049bba8b105d None admin] PATH_INFO: `/v3/projects/0729fa47cb4b41098219d643a030381c` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:24.248458 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a54d63-23c0-4bb5-a671-049bba8b105d None admin] RBAC: Authorizing `identity:get_project(project_id=0729fa47cb4b41098219d643a030381c)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:02:24.251265 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a54d63-23c0-4bb5-a671-049bba8b105d None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:02:24.254651 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 580/1152] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 2 19:02:24 2022] GET /identity/v3/projects/0729fa47cb4b41098219d643a030381c => generated 311 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:02:24.261651 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0212097e-78bf-4502-8e9c-adc227046759 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:02:24.321040 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:02:24.370531 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0212097e-78bf-4502-8e9c-adc227046759 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:02:24.406962 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4728f331-26ad-48c2-b0ae-47a87eae44d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:02:24.407858 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4728f331-26ad-48c2-b0ae-47a87eae44d6 None admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:24.408148 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4728f331-26ad-48c2-b0ae-47a87eae44d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:24.408423 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4728f331-26ad-48c2-b0ae-47a87eae44d6 None admin] PATH_INFO: `/v3/projects/0729fa47cb4b41098219d643a030381c/users/19a8f99f6f27466aa99a6a23e3952c91/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:24.410403 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4728f331-26ad-48c2-b0ae-47a87eae44d6 None admin] RBAC: Authorizing `identity:create_grant(project_id=0729fa47cb4b41098219d643a030381c, user_id=19a8f99f6f27466aa99a6a23e3952c91, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:02:24.411877 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4728f331-26ad-48c2-b0ae-47a87eae44d6 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:02:24.503985 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 573/1153] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 2 19:02:24 2022] PUT /identity/v3/projects/0729fa47cb4b41098219d643a030381c/users/19a8f99f6f27466aa99a6a23e3952c91/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:02:26.853308 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e4f6f4-25a0-4304-8511-ae504999e41d None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:26.853902 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e4f6f4-25a0-4304-8511-ae504999e41d None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:26.853902 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e4f6f4-25a0-4304-8511-ae504999e41d None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:26.855456 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 581/1154] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:02:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:02:26.861416 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b02a3c4-a46a-48e6-bf98-c79bb71e613c None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:26.862030 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b02a3c4-a46a-48e6-bf98-c79bb71e613c None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:26.862157 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b02a3c4-a46a-48e6-bf98-c79bb71e613c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:26.879391 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-3b02a3c4-a46a-48e6-bf98-c79bb71e613c None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:02:26.880625 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-3b02a3c4-a46a-48e6-bf98-c79bb71e613c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:02:26.952466 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 574/1155] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:02:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 02 19:02:26.976575 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e4f6f4-25a0-4304-8511-ae504999e41d None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:02:27.084375 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e4f6f4-25a0-4304-8511-ae504999e41d None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:02:27.110176 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:02:27.128912 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdc9625-a980-42e1-80b0-08dfc6f8d5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:02:27.129825 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc9625-a980-42e1-80b0-08dfc6f8d5d1 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:27.130238 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc9625-a980-42e1-80b0-08dfc6f8d5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:27.130573 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc9625-a980-42e1-80b0-08dfc6f8d5d1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:27.145409 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdc9625-a980-42e1-80b0-08dfc6f8d5d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:02:27.154698 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-9fdc9625-a980-42e1-80b0-08dfc6f8d5d1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 02 19:02:27.154698 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:02:27.154698 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:02:27.154698 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:02:27.154698 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:02:27.154698 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:02:27.154698 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:02:27.154698 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:02:27.154698 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:02:27.154698 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:02:27.154698 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:02:27.154698 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:02:27.154698 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:02:27.154698 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:02:27.154698 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:02:27.154698 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:02:27.156650 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:02:27.156650 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:02:27.156650 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:02:27.156650 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:02:27.156650 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:02:27.156650 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:02:27.156650 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:02:27.156650 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:02:27.156650 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:02:27.156650 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:02:27.156650 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:02:27.156650 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:02:27.156650 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:02:27.156650 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:02:27.156650 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:02:27.156650 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:02:27.158367 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:02:27.158367 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:02:27.158367 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:02:27.158367 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:02:27.158367 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:02:27.158367 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:02:27.158367 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:02:27.158367 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:02:27.158367 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:02:27.158367 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:02:27.158367 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 02 19:02:27.158367 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:02:27.158367 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 582/1156] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 2 19:02:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 02 19:02:27.162355 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b02a3c4-a46a-48e6-bf98-c79bb71e613c None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:02:27.209055 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:02:27.274836 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b02a3c4-a46a-48e6-bf98-c79bb71e613c None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:02:27.317528 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b4c32f-292a-4bdd-85a7-ff720a687ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:02:27.318202 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b4c32f-292a-4bdd-85a7-ff720a687ea8 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:27.318489 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b4c32f-292a-4bdd-85a7-ff720a687ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:27.318791 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b4c32f-292a-4bdd-85a7-ff720a687ea8 None admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:27.319601 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b4c32f-292a-4bdd-85a7-ff720a687ea8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:02:27.319911 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b4c32f-292a-4bdd-85a7-ff720a687ea8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:02:27.321387 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b4c32f-292a-4bdd-85a7-ff720a687ea8 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:02:27.332377 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 575/1157] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 2 19:02:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:02:27.339303 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdc9625-a980-42e1-80b0-08dfc6f8d5d1 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:02:27.456623 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdc9625-a980-42e1-80b0-08dfc6f8d5d1 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:02:27.494018 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb34c764-6197-4db3-9797-812250dfac01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:02:27.494990 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34c764-6197-4db3-9797-812250dfac01 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:27.495315 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34c764-6197-4db3-9797-812250dfac01 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:27.495633 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34c764-6197-4db3-9797-812250dfac01 None admin] PATH_INFO: `/v3/users/19a8f99f6f27466aa99a6a23e3952c91` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:27.496985 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb34c764-6197-4db3-9797-812250dfac01 None admin] RBAC: Authorizing `identity:get_user(user_id=19a8f99f6f27466aa99a6a23e3952c91)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:02:27.498488 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb34c764-6197-4db3-9797-812250dfac01 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:02:27.500264 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 583/1158] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 2 19:02:27 2022] GET /identity/v3/users/19a8f99f6f27466aa99a6a23e3952c91 => generated 286 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:02:27.506355 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b4c32f-292a-4bdd-85a7-ff720a687ea8 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:02:27.621823 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b4c32f-292a-4bdd-85a7-ff720a687ea8 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:02:27.624827 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:02:27.663421 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d701cb11-7dab-42c0-9a63-83ac5f5fdaa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:02:27.664092 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d701cb11-7dab-42c0-9a63-83ac5f5fdaa8 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:27.664379 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d701cb11-7dab-42c0-9a63-83ac5f5fdaa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:27.664657 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d701cb11-7dab-42c0-9a63-83ac5f5fdaa8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:27.747718 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d701cb11-7dab-42c0-9a63-83ac5f5fdaa8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:02:27.749232 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d701cb11-7dab-42c0-9a63-83ac5f5fdaa8 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:02:27.862391 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 576/1159] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 2 19:02:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 02 19:02:27.868664 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb34c764-6197-4db3-9797-812250dfac01 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:02:27.923138 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:02:27.978273 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb34c764-6197-4db3-9797-812250dfac01 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:02:28.017064 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80f1e79-948d-40e1-8a8e-eeb8d656ded2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:02:28.017860 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f1e79-948d-40e1-8a8e-eeb8d656ded2 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:28.018147 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f1e79-948d-40e1-8a8e-eeb8d656ded2 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:28.018429 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f1e79-948d-40e1-8a8e-eeb8d656ded2 None admin] PATH_INFO: `/v3/projects/0729fa47cb4b41098219d643a030381c` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:28.019695 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80f1e79-948d-40e1-8a8e-eeb8d656ded2 None admin] RBAC: Authorizing `identity:get_project(project_id=0729fa47cb4b41098219d643a030381c)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:02:28.021151 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80f1e79-948d-40e1-8a8e-eeb8d656ded2 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:02:28.022917 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 584/1160] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 2 19:02:27 2022] GET /identity/v3/projects/0729fa47cb4b41098219d643a030381c => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:02:28.029332 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d701cb11-7dab-42c0-9a63-83ac5f5fdaa8 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:02:28.032594 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:02:28.141072 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d701cb11-7dab-42c0-9a63-83ac5f5fdaa8 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:02:28.182944 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c1ba0a-b24e-4a2f-ab30-ad766c586cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:02:28.183618 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c1ba0a-b24e-4a2f-ab30-ad766c586cad None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:28.183908 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c1ba0a-b24e-4a2f-ab30-ad766c586cad None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:28.184188 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c1ba0a-b24e-4a2f-ab30-ad766c586cad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:28.185038 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c1ba0a-b24e-4a2f-ab30-ad766c586cad None admin] RBAC: Adding query filter params (role.id=f13fdd0847924014abf1e95594195e78, scope.project.id=0729fa47cb4b41098219d643a030381c, user.id=19a8f99f6f27466aa99a6a23e3952c91) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:02:28.185356 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c1ba0a-b24e-4a2f-ab30-ad766c586cad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:02:28.186858 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c1ba0a-b24e-4a2f-ab30-ad766c586cad None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:02:28.193510 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 577/1161] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 2 19:02:28 2022] GET /identity/v3/role_assignments?user.id=19a8f99f6f27466aa99a6a23e3952c91&scope.project.id=0729fa47cb4b41098219d643a030381c&role.id=f13fdd0847924014abf1e95594195e78 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:02:30.576132 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9850348c-d961-41d4-8e11-f94d0cc6c444 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:30.576709 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9850348c-d961-41d4-8e11-f94d0cc6c444 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:30.576709 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9850348c-d961-41d4-8e11-f94d0cc6c444 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:30.578049 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 585/1162] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:02:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:02:30.583722 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f31fd-f15d-44d3-9d76-30efaf594ab3 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:30.584178 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f31fd-f15d-44d3-9d76-30efaf594ab3 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:30.584274 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f31fd-f15d-44d3-9d76-30efaf594ab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:30.603952 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-070f31fd-f15d-44d3-9d76-30efaf594ab3 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:02:30.605211 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-070f31fd-f15d-44d3-9d76-30efaf594ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:02:30.669441 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:02:30.674921 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 578/1163] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:02:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 02 19:02:30.698897 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9850348c-d961-41d4-8e11-f94d0cc6c444 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:02:30.805119 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9850348c-d961-41d4-8e11-f94d0cc6c444 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:02:30.826877 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:02:30.845954 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f344f543-40a5-4cb3-a1a1-25530f95775b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:02:30.846701 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f344f543-40a5-4cb3-a1a1-25530f95775b None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:30.847243 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f344f543-40a5-4cb3-a1a1-25530f95775b None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:30.847530 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f344f543-40a5-4cb3-a1a1-25530f95775b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:30.921587 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f344f543-40a5-4cb3-a1a1-25530f95775b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:02:30.923115 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f344f543-40a5-4cb3-a1a1-25530f95775b None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:02:31.027890 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 586/1164] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 2 19:02:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 02 19:02:31.034412 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070f31fd-f15d-44d3-9d76-30efaf594ab3 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:02:31.153395 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070f31fd-f15d-44d3-9d76-30efaf594ab3 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:02:31.189964 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b949be5-8a36-4121-8541-bdbd50269e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:02:31.190795 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b949be5-8a36-4121-8541-bdbd50269e01 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:31.191084 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b949be5-8a36-4121-8541-bdbd50269e01 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:31.191359 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b949be5-8a36-4121-8541-bdbd50269e01 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:31.198773 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b949be5-8a36-4121-8541-bdbd50269e01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:02:31.200238 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b949be5-8a36-4121-8541-bdbd50269e01 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:02:31.214830 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-3b949be5-8a36-4121-8541-bdbd50269e01 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 02 19:02:31.214830 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:02:31.214830 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 02 19:02:31.214830 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 02 19:02:31.214830 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 02 19:02:31.214830 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 02 19:02:31.214830 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 02 19:02:31.214830 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 02 19:02:31.214830 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 02 19:02:31.214830 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:02:31.214830 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 02 19:02:31.214830 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:02:31.214830 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:02:31.214830 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:02:31.214830 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:02:31.214830 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:02:31.214830 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:02:31.216842 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:02:31.216842 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:02:31.216842 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:02:31.216842 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:02:31.216842 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:02:31.216842 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:02:31.216842 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:02:31.216842 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:02:31.216842 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:02:31.216842 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:02:31.216842 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:02:31.216842 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:02:31.216842 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:02:31.216842 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:02:31.216842 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:02:31.216842 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:02:31.218653 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:02:31.218653 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:02:31.218653 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:02:31.218653 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:02:31.218653 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:02:31.218653 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:02:31.218653 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:02:31.218653 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:02:31.218653 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:02:31.218653 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:02:31.218653 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 02 19:02:31.218653 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:02:31.218653 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 02 19:02:31.218653 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:02:31.218653 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 579/1165] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Jun 2 19:02:31 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:02:31.224041 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f344f543-40a5-4cb3-a1a1-25530f95775b None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:02:31.243968 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:02:31.336653 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f344f543-40a5-4cb3-a1a1-25530f95775b None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:02:31.373163 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7748e9ad-eab5-4602-ab89-5f4ae0fb5c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:02:31.373753 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7748e9ad-eab5-4602-ab89-5f4ae0fb5c36 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:31.374041 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7748e9ad-eab5-4602-ab89-5f4ae0fb5c36 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:31.374323 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7748e9ad-eab5-4602-ab89-5f4ae0fb5c36 None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:31.375095 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7748e9ad-eab5-4602-ab89-5f4ae0fb5c36 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:02:31.375412 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7748e9ad-eab5-4602-ab89-5f4ae0fb5c36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:02:31.376798 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7748e9ad-eab5-4602-ab89-5f4ae0fb5c36 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:02:31.385144 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 587/1166] 10.222.0.33 () {56 vars in 1115 bytes} [Thu Jun 2 19:02:31 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:02:33.789099 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef455712-5a1a-4ae4-89ef-89f652f1e1d3 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:33.789706 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef455712-5a1a-4ae4-89ef-89f652f1e1d3 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:33.789706 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef455712-5a1a-4ae4-89ef-89f652f1e1d3 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:33.791151 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 580/1167] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:02:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 02 19:02:33.797474 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c7291-e4aa-49d0-9f31-b886d55a6028 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:33.798045 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c7291-e4aa-49d0-9f31-b886d55a6028 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:33.798214 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c7291-e4aa-49d0-9f31-b886d55a6028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:33.816877 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-940c7291-e4aa-49d0-9f31-b886d55a6028 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:02:33.818255 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-940c7291-e4aa-49d0-9f31-b886d55a6028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:02:33.898996 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 588/1168] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:02:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 02 19:02:33.924215 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef455712-5a1a-4ae4-89ef-89f652f1e1d3 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:02:33.990450 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:02:34.035646 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef455712-5a1a-4ae4-89ef-89f652f1e1d3 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:02:34.072561 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4e7721-35f1-4717-977b-ab8522223532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:02:34.073185 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4e7721-35f1-4717-977b-ab8522223532 None admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:34.073469 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4e7721-35f1-4717-977b-ab8522223532 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:34.073749 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4e7721-35f1-4717-977b-ab8522223532 None admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:34.074517 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4e7721-35f1-4717-977b-ab8522223532 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:02:34.075959 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4e7721-35f1-4717-977b-ab8522223532 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:02:34.162169 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 581/1169] 10.222.0.33 () {60 vars in 1138 bytes} [Thu Jun 2 19:02:33 2022] POST /identity/v3/domains => generated 267 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 02 19:02:36.651009 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d88a335-1250-4dd6-b655-3ae9966f5852 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:36.651868 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d88a335-1250-4dd6-b655-3ae9966f5852 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:36.652052 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d88a335-1250-4dd6-b655-3ae9966f5852 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:36.653954 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 589/1170] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:02:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 02 19:02:36.659814 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d152b07d-033d-4bda-bd6a-d191ba7fa329 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:36.660303 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d152b07d-033d-4bda-bd6a-d191ba7fa329 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:36.660456 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d152b07d-033d-4bda-bd6a-d191ba7fa329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:36.677562 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-d152b07d-033d-4bda-bd6a-d191ba7fa329 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:02:36.678903 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-d152b07d-033d-4bda-bd6a-d191ba7fa329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:02:36.751777 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 582/1171] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:02:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 02 19:02:36.776884 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d88a335-1250-4dd6-b655-3ae9966f5852 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:02:36.793117 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:02:36.886556 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d88a335-1250-4dd6-b655-3ae9966f5852 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:02:36.921123 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e26059-6f0c-434d-bb16-cdcfb693e1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:02:36.921911 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e26059-6f0c-434d-bb16-cdcfb693e1d0 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:36.922194 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e26059-6f0c-434d-bb16-cdcfb693e1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:36.922469 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e26059-6f0c-434d-bb16-cdcfb693e1d0 None admin] PATH_INFO: `/v3/domains/2510f199ed814c489a12ed99687902b7` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:36.934060 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e26059-6f0c-434d-bb16-cdcfb693e1d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=2510f199ed814c489a12ed99687902b7)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:02:36.935556 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e26059-6f0c-434d-bb16-cdcfb693e1d0 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:02:36.937299 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 590/1172] 10.222.0.33 () {56 vars in 1183 bytes} [Thu Jun 2 19:02:36 2022] GET /identity/v3/domains/2510f199ed814c489a12ed99687902b7 => generated 267 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:02:36.943404 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d152b07d-033d-4bda-bd6a-d191ba7fa329 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:02:36.997906 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:02:37.052863 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d152b07d-033d-4bda-bd6a-d191ba7fa329 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:02:37.091542 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3d43e1-467c-4ff7-81be-d3db55622ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:02:37.092211 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3d43e1-467c-4ff7-81be-d3db55622ed5 None admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:37.092496 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3d43e1-467c-4ff7-81be-d3db55622ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:37.092767 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3d43e1-467c-4ff7-81be-d3db55622ed5 None admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:37.093495 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3d43e1-467c-4ff7-81be-d3db55622ed5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:02:37.094961 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3d43e1-467c-4ff7-81be-d3db55622ed5 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:02:37.201874 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 583/1173] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Jun 2 19:02:36 2022] POST /identity/v3/projects => generated 361 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 02 19:02:39.557456 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f9a228-344f-4040-a839-75a3c7ffe944 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:39.558571 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f9a228-344f-4040-a839-75a3c7ffe944 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:39.558571 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f9a228-344f-4040-a839-75a3c7ffe944 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:39.559974 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 591/1174] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:02:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:02:39.565610 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cc18db-1b37-448c-9cab-64e471c317d3 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:39.566105 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cc18db-1b37-448c-9cab-64e471c317d3 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:39.566209 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cc18db-1b37-448c-9cab-64e471c317d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:39.582949 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-d1cc18db-1b37-448c-9cab-64e471c317d3 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:02:39.584138 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-d1cc18db-1b37-448c-9cab-64e471c317d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:02:39.651616 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 584/1175] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:02:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 02 19:02:39.677396 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f9a228-344f-4040-a839-75a3c7ffe944 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:02:39.750950 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:02:39.785401 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f9a228-344f-4040-a839-75a3c7ffe944 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:02:39.820857 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ac7d51-4adf-4fd3-aa60-56b7abea95f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:02:39.821640 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ac7d51-4adf-4fd3-aa60-56b7abea95f9 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:39.821927 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ac7d51-4adf-4fd3-aa60-56b7abea95f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:39.822206 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ac7d51-4adf-4fd3-aa60-56b7abea95f9 None admin] PATH_INFO: `/v3/domains/2510f199ed814c489a12ed99687902b7` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:39.824023 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ac7d51-4adf-4fd3-aa60-56b7abea95f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=2510f199ed814c489a12ed99687902b7)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:02:39.825489 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ac7d51-4adf-4fd3-aa60-56b7abea95f9 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:02:39.827219 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 592/1176] 10.222.0.33 () {56 vars in 1183 bytes} [Thu Jun 2 19:02:39 2022] GET /identity/v3/domains/2510f199ed814c489a12ed99687902b7 => generated 267 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:02:39.833090 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cc18db-1b37-448c-9cab-64e471c317d3 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:02:39.864232 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:02:39.936197 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cc18db-1b37-448c-9cab-64e471c317d3 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:02:39.971345 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9c139b-44e5-4e30-9025-b2024deade48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:02:39.972020 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c139b-44e5-4e30-9025-b2024deade48 None admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:39.972302 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c139b-44e5-4e30-9025-b2024deade48 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:39.972584 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c139b-44e5-4e30-9025-b2024deade48 None admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:39.973298 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9c139b-44e5-4e30-9025-b2024deade48 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:02:39.974729 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9c139b-44e5-4e30-9025-b2024deade48 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:02:40.041185 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 585/1177] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 2 19:02:39 2022] POST /identity/v3/users => generated 311 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:02:42.429609 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d7debe-08dd-4744-b14a-05dde76851c9 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:42.430129 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d7debe-08dd-4744-b14a-05dde76851c9 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:42.430129 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d7debe-08dd-4744-b14a-05dde76851c9 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:42.431753 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 593/1178] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:02:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 02 19:02:42.437690 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6128cd7-83ca-49aa-a57e-e41199d48f6e None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:42.438215 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6128cd7-83ca-49aa-a57e-e41199d48f6e None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:42.438325 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6128cd7-83ca-49aa-a57e-e41199d48f6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:42.456527 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-c6128cd7-83ca-49aa-a57e-e41199d48f6e None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:02:42.457754 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-c6128cd7-83ca-49aa-a57e-e41199d48f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:02:42.529266 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 586/1179] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:02:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 02 19:02:42.553575 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d7debe-08dd-4744-b14a-05dde76851c9 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:02:42.664419 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d7debe-08dd-4744-b14a-05dde76851c9 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:02:42.701217 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9758307-d4db-499e-ab99-76ebdde5cffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:02:42.701935 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9758307-d4db-499e-ab99-76ebdde5cffe None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:42.702217 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9758307-d4db-499e-ab99-76ebdde5cffe None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:42.702488 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9758307-d4db-499e-ab99-76ebdde5cffe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:42.710921 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:02:42.715719 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9758307-d4db-499e-ab99-76ebdde5cffe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:02:42.723677 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-b9758307-d4db-499e-ab99-76ebdde5cffe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 02 19:02:42.723677 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:02:42.723677 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:02:42.723677 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:02:42.723677 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:02:42.723677 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:02:42.723677 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:02:42.723677 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:02:42.723677 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:02:42.723677 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:02:42.723677 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:02:42.723677 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:02:42.723677 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:02:42.723677 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:02:42.723677 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:02:42.723677 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:02:42.729685 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:02:42.729685 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:02:42.729685 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:02:42.729685 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:02:42.729685 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:02:42.729685 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:02:42.729685 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:02:42.729685 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:02:42.729685 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:02:42.729685 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:02:42.729685 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:02:42.729685 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:02:42.729685 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:02:42.729685 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:02:42.729685 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:02:42.729685 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:02:42.731349 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:02:42.731349 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:02:42.731349 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:02:42.731349 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:02:42.731349 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:02:42.731349 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:02:42.731349 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:02:42.731349 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:02:42.731349 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:02:42.731349 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:02:42.731349 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 02 19:02:42.731349 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:02:42.731349 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 594/1180] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 2 19:02:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:02:42.732681 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6128cd7-83ca-49aa-a57e-e41199d48f6e None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:02:42.754631 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:02:42.854596 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6128cd7-83ca-49aa-a57e-e41199d48f6e None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:02:42.894556 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafa509f-9c21-44eb-81c3-613df7e29168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:02:42.895349 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafa509f-9c21-44eb-81c3-613df7e29168 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:42.895685 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafa509f-9c21-44eb-81c3-613df7e29168 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:42.896038 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafa509f-9c21-44eb-81c3-613df7e29168 None admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:42.896940 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafa509f-9c21-44eb-81c3-613df7e29168 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:02:42.897302 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafa509f-9c21-44eb-81c3-613df7e29168 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:02:42.898853 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafa509f-9c21-44eb-81c3-613df7e29168 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:02:42.909464 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 587/1181] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 2 19:02:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:02:42.916019 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9758307-d4db-499e-ab99-76ebdde5cffe None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:02:43.025683 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9758307-d4db-499e-ab99-76ebdde5cffe None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:02:43.061652 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e71b2fc-15f7-47c9-8d8b-60679b17d18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:02:43.062553 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71b2fc-15f7-47c9-8d8b-60679b17d18b None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:43.062875 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71b2fc-15f7-47c9-8d8b-60679b17d18b None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:43.063161 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71b2fc-15f7-47c9-8d8b-60679b17d18b None admin] PATH_INFO: `/v3/users/257c9459782d4c2ca243f08ed6f163cf` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:43.078442 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e71b2fc-15f7-47c9-8d8b-60679b17d18b None admin] RBAC: Authorizing `identity:get_user(user_id=257c9459782d4c2ca243f08ed6f163cf)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:02:43.079972 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e71b2fc-15f7-47c9-8d8b-60679b17d18b None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:02:43.081774 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 595/1182] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 2 19:02:42 2022] GET /identity/v3/users/257c9459782d4c2ca243f08ed6f163cf => generated 311 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:02:43.091153 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafa509f-9c21-44eb-81c3-613df7e29168 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:02:43.184206 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:02:43.207704 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafa509f-9c21-44eb-81c3-613df7e29168 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:02:43.256290 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7a15c5-9f6e-49cd-bdc6-5bf2aec20863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:02:43.256951 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a15c5-9f6e-49cd-bdc6-5bf2aec20863 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:43.257235 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a15c5-9f6e-49cd-bdc6-5bf2aec20863 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:43.257510 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a15c5-9f6e-49cd-bdc6-5bf2aec20863 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:43.335808 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7a15c5-9f6e-49cd-bdc6-5bf2aec20863 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:02:43.337224 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7a15c5-9f6e-49cd-bdc6-5bf2aec20863 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:02:43.426701 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:02:43.449837 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 588/1183] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 2 19:02:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 02 19:02:43.455905 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e71b2fc-15f7-47c9-8d8b-60679b17d18b None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:02:43.521092 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:02:43.565481 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e71b2fc-15f7-47c9-8d8b-60679b17d18b None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:02:43.602365 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952d2dde-20ea-4d21-a2b5-9fb3e2ab5493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:02:43.603238 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952d2dde-20ea-4d21-a2b5-9fb3e2ab5493 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:43.603570 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952d2dde-20ea-4d21-a2b5-9fb3e2ab5493 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:43.603894 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952d2dde-20ea-4d21-a2b5-9fb3e2ab5493 None admin] PATH_INFO: `/v3/projects/712b019509924e58b3f25b0aca4d7288` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:43.605776 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952d2dde-20ea-4d21-a2b5-9fb3e2ab5493 None admin] RBAC: Authorizing `identity:get_project(project_id=712b019509924e58b3f25b0aca4d7288)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:02:43.607341 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952d2dde-20ea-4d21-a2b5-9fb3e2ab5493 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:02:43.609156 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 596/1184] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 2 19:02:43 2022] GET /identity/v3/projects/712b019509924e58b3f25b0aca4d7288 => generated 361 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:02:43.615652 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7a15c5-9f6e-49cd-bdc6-5bf2aec20863 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:02:43.724916 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7a15c5-9f6e-49cd-bdc6-5bf2aec20863 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:02:43.763592 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13241f56-fcce-4a77-86e9-615ebdb07494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:02:43.764361 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13241f56-fcce-4a77-86e9-615ebdb07494 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:43.764684 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13241f56-fcce-4a77-86e9-615ebdb07494 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:43.765074 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13241f56-fcce-4a77-86e9-615ebdb07494 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:43.765967 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13241f56-fcce-4a77-86e9-615ebdb07494 None admin] RBAC: Adding query filter params (role.id=f13fdd0847924014abf1e95594195e78, scope.project.id=712b019509924e58b3f25b0aca4d7288, user.id=257c9459782d4c2ca243f08ed6f163cf) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:02:43.766326 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13241f56-fcce-4a77-86e9-615ebdb07494 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:02:43.767874 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13241f56-fcce-4a77-86e9-615ebdb07494 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:02:43.774105 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 589/1185] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 2 19:02:43 2022] GET /identity/v3/role_assignments?user.id=257c9459782d4c2ca243f08ed6f163cf&scope.project.id=712b019509924e58b3f25b0aca4d7288&role.id=f13fdd0847924014abf1e95594195e78 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:02:46.081173 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e766d56e-ee17-46b8-a887-b9cc30025ecb None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:46.081746 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e766d56e-ee17-46b8-a887-b9cc30025ecb None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:46.081746 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e766d56e-ee17-46b8-a887-b9cc30025ecb None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:46.083168 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 597/1186] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:02:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 02 19:02:46.088668 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d17a7bf-a53d-47ac-a188-66cfc57a6580 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:46.089154 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d17a7bf-a53d-47ac-a188-66cfc57a6580 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:46.089273 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d17a7bf-a53d-47ac-a188-66cfc57a6580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:46.106448 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-4d17a7bf-a53d-47ac-a188-66cfc57a6580 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:02:46.107705 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-4d17a7bf-a53d-47ac-a188-66cfc57a6580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:02:46.145069 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:02:46.176077 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 590/1187] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:02:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 02 19:02:46.199633 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e766d56e-ee17-46b8-a887-b9cc30025ecb None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:02:46.314242 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e766d56e-ee17-46b8-a887-b9cc30025ecb None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:02:46.345467 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:02:46.355837 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67cb914-4d48-464e-8bb5-1c9bc5e7466b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:02:46.356754 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67cb914-4d48-464e-8bb5-1c9bc5e7466b None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:46.357196 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67cb914-4d48-464e-8bb5-1c9bc5e7466b None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:46.357484 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67cb914-4d48-464e-8bb5-1c9bc5e7466b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:46.369620 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67cb914-4d48-464e-8bb5-1c9bc5e7466b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:02:46.378222 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-e67cb914-4d48-464e-8bb5-1c9bc5e7466b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 02 19:02:46.378222 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:02:46.378222 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:02:46.378222 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:02:46.378222 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:02:46.378222 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:02:46.378222 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:02:46.378222 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:02:46.378222 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:02:46.378222 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:02:46.378222 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:02:46.378222 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:02:46.378222 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:02:46.378222 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:02:46.378222 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:02:46.378222 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:02:46.379507 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:02:46.379507 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:02:46.379507 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:02:46.379507 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:02:46.379507 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:02:46.379507 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:02:46.379507 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:02:46.379507 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:02:46.379507 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:02:46.379507 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:02:46.379507 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:02:46.379507 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:02:46.379507 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:02:46.379507 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:02:46.379507 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:02:46.379507 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:02:46.381041 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:02:46.381041 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:02:46.381041 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:02:46.381041 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:02:46.381041 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:02:46.381041 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:02:46.381041 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:02:46.381041 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:02:46.381041 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:02:46.381041 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:02:46.381041 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 02 19:02:46.381041 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:02:46.381041 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 598/1188] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 2 19:02:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:02:46.387973 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d17a7bf-a53d-47ac-a188-66cfc57a6580 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:02:46.496938 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d17a7bf-a53d-47ac-a188-66cfc57a6580 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:02:46.532718 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a5eb2b-aded-49f8-9c71-d767f0ce73e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:02:46.533387 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a5eb2b-aded-49f8-9c71-d767f0ce73e8 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:46.533675 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a5eb2b-aded-49f8-9c71-d767f0ce73e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:46.533962 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a5eb2b-aded-49f8-9c71-d767f0ce73e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:46.534722 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a5eb2b-aded-49f8-9c71-d767f0ce73e8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:02:46.535062 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a5eb2b-aded-49f8-9c71-d767f0ce73e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:02:46.536464 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a5eb2b-aded-49f8-9c71-d767f0ce73e8 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:02:46.545562 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 591/1189] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 2 19:02:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:02:46.551834 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67cb914-4d48-464e-8bb5-1c9bc5e7466b None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:02:46.658009 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67cb914-4d48-464e-8bb5-1c9bc5e7466b None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:02:46.693933 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d548e3b-26da-44e7-b0b5-62e7155e4dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:02:46.694839 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d548e3b-26da-44e7-b0b5-62e7155e4dea None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:46.695125 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d548e3b-26da-44e7-b0b5-62e7155e4dea None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:46.695398 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d548e3b-26da-44e7-b0b5-62e7155e4dea None admin] PATH_INFO: `/v3/users/257c9459782d4c2ca243f08ed6f163cf` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:46.696616 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d548e3b-26da-44e7-b0b5-62e7155e4dea None admin] RBAC: Authorizing `identity:get_user(user_id=257c9459782d4c2ca243f08ed6f163cf)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:02:46.698087 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d548e3b-26da-44e7-b0b5-62e7155e4dea None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:02:46.699816 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 599/1190] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 2 19:02:46 2022] GET /identity/v3/users/257c9459782d4c2ca243f08ed6f163cf => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:02:46.705677 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a5eb2b-aded-49f8-9c71-d767f0ce73e8 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:02:46.742820 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:02:46.857342 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a5eb2b-aded-49f8-9c71-d767f0ce73e8 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:02:46.912961 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a14059-698a-4d23-b030-19858e10c1fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:02:46.914397 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a14059-698a-4d23-b030-19858e10c1fe None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:46.915030 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a14059-698a-4d23-b030-19858e10c1fe None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:46.915537 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a14059-698a-4d23-b030-19858e10c1fe None admin] PATH_INFO: `/v3/projects/712b019509924e58b3f25b0aca4d7288` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:46.918266 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a14059-698a-4d23-b030-19858e10c1fe None admin] RBAC: Authorizing `identity:get_project(project_id=712b019509924e58b3f25b0aca4d7288)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:02:46.921024 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a14059-698a-4d23-b030-19858e10c1fe None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:02:46.924279 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 592/1191] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 2 19:02:46 2022] GET /identity/v3/projects/712b019509924e58b3f25b0aca4d7288 => generated 361 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:02:46.930527 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d548e3b-26da-44e7-b0b5-62e7155e4dea None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:02:46.992698 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:02:47.052382 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d548e3b-26da-44e7-b0b5-62e7155e4dea None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:02:47.087396 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada4008d-7a2f-400d-a9ee-0b54e6dd5ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:02:47.088292 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada4008d-7a2f-400d-a9ee-0b54e6dd5ecf None admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:47.088580 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada4008d-7a2f-400d-a9ee-0b54e6dd5ecf None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:47.088860 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada4008d-7a2f-400d-a9ee-0b54e6dd5ecf None admin] PATH_INFO: `/v3/projects/712b019509924e58b3f25b0aca4d7288/users/257c9459782d4c2ca243f08ed6f163cf/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:47.090993 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada4008d-7a2f-400d-a9ee-0b54e6dd5ecf None admin] RBAC: Authorizing `identity:create_grant(project_id=712b019509924e58b3f25b0aca4d7288, user_id=257c9459782d4c2ca243f08ed6f163cf, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:02:47.092470 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada4008d-7a2f-400d-a9ee-0b54e6dd5ecf None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:02:47.167121 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 600/1192] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 2 19:02:46 2022] PUT /identity/v3/projects/712b019509924e58b3f25b0aca4d7288/users/257c9459782d4c2ca243f08ed6f163cf/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 02 19:02:49.818689 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac81bd5-ddc7-4d1e-aaef-e7e5ec0dff29 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:49.819319 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac81bd5-ddc7-4d1e-aaef-e7e5ec0dff29 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:49.819319 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac81bd5-ddc7-4d1e-aaef-e7e5ec0dff29 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:49.820668 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 593/1193] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:02:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:02:49.826834 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d44186-b028-479e-83f6-7e39c6c0fec2 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:49.827618 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d44186-b028-479e-83f6-7e39c6c0fec2 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:49.827618 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d44186-b028-479e-83f6-7e39c6c0fec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:49.846200 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-31d44186-b028-479e-83f6-7e39c6c0fec2 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:02:49.847532 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-31d44186-b028-479e-83f6-7e39c6c0fec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:02:49.918452 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 601/1194] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:02:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 02 19:02:49.943593 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac81bd5-ddc7-4d1e-aaef-e7e5ec0dff29 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:02:50.066052 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac81bd5-ddc7-4d1e-aaef-e7e5ec0dff29 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:02:50.067894 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:02:50.105323 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bf03bf-c3aa-4904-b055-4757483ecc34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:02:50.106120 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bf03bf-c3aa-4904-b055-4757483ecc34 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:50.106453 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bf03bf-c3aa-4904-b055-4757483ecc34 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:50.106808 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bf03bf-c3aa-4904-b055-4757483ecc34 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:50.122491 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bf03bf-c3aa-4904-b055-4757483ecc34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:02:50.130442 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-77bf03bf-c3aa-4904-b055-4757483ecc34 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 02 19:02:50.130442 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:02:50.130442 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:02:50.130442 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:02:50.130442 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:02:50.130442 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:02:50.130442 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:02:50.130442 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:02:50.130442 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:02:50.130442 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:02:50.130442 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:02:50.130442 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:02:50.130442 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:02:50.130442 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:02:50.130442 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:02:50.130442 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:02:50.132317 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:02:50.132317 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:02:50.132317 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:02:50.132317 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:02:50.132317 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:02:50.132317 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:02:50.132317 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:02:50.132317 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:02:50.132317 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:02:50.132317 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:02:50.132317 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:02:50.132317 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:02:50.132317 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:02:50.132317 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:02:50.132317 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:02:50.132317 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:02:50.134261 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:02:50.134261 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:02:50.134261 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:02:50.134261 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:02:50.134261 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:02:50.134261 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:02:50.134261 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:02:50.134261 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:02:50.134261 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:02:50.134261 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:02:50.134261 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 02 19:02:50.134261 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:02:50.134261 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 594/1195] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 2 19:02:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 02 19:02:50.142682 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d44186-b028-479e-83f6-7e39c6c0fec2 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:02:50.224456 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:02:50.283175 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d44186-b028-479e-83f6-7e39c6c0fec2 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:02:50.320351 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9713f795-2150-4705-9248-adc4aaec3042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:02:50.321084 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9713f795-2150-4705-9248-adc4aaec3042 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:50.321415 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9713f795-2150-4705-9248-adc4aaec3042 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:50.321741 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9713f795-2150-4705-9248-adc4aaec3042 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:50.322551 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9713f795-2150-4705-9248-adc4aaec3042 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:02:50.322959 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9713f795-2150-4705-9248-adc4aaec3042 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:02:50.324990 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9713f795-2150-4705-9248-adc4aaec3042 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:02:50.334203 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 602/1196] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 2 19:02:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:02:50.340439 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bf03bf-c3aa-4904-b055-4757483ecc34 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:02:50.451609 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bf03bf-c3aa-4904-b055-4757483ecc34 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:02:50.487224 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e341d0-e242-4e58-a861-0112b9838908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:02:50.488106 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e341d0-e242-4e58-a861-0112b9838908 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:50.488394 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e341d0-e242-4e58-a861-0112b9838908 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:50.488675 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e341d0-e242-4e58-a861-0112b9838908 None admin] PATH_INFO: `/v3/users/257c9459782d4c2ca243f08ed6f163cf` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:50.489934 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e341d0-e242-4e58-a861-0112b9838908 None admin] RBAC: Authorizing `identity:get_user(user_id=257c9459782d4c2ca243f08ed6f163cf)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:02:50.491435 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e341d0-e242-4e58-a861-0112b9838908 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:02:50.493127 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 595/1197] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Jun 2 19:02:50 2022] GET /identity/v3/users/257c9459782d4c2ca243f08ed6f163cf => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:02:50.499810 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9713f795-2150-4705-9248-adc4aaec3042 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:02:50.608373 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9713f795-2150-4705-9248-adc4aaec3042 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:02:50.616896 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:02:50.648490 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903a8f22-7fbf-446d-a813-3b63dba8c133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:02:50.649165 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a8f22-7fbf-446d-a813-3b63dba8c133 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:50.649455 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a8f22-7fbf-446d-a813-3b63dba8c133 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:50.649737 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a8f22-7fbf-446d-a813-3b63dba8c133 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:50.722127 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903a8f22-7fbf-446d-a813-3b63dba8c133 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:02:50.723639 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903a8f22-7fbf-446d-a813-3b63dba8c133 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:02:50.837100 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 603/1198] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 2 19:02:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 02 19:02:50.845179 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e341d0-e242-4e58-a861-0112b9838908 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:02:50.882108 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:02:50.961108 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e341d0-e242-4e58-a861-0112b9838908 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:02:51.002664 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff955a98-b0af-455e-ae48-6c402d6da46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:02:51.003491 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff955a98-b0af-455e-ae48-6c402d6da46e None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:51.003782 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff955a98-b0af-455e-ae48-6c402d6da46e None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:51.004065 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff955a98-b0af-455e-ae48-6c402d6da46e None admin] PATH_INFO: `/v3/projects/712b019509924e58b3f25b0aca4d7288` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:51.005342 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff955a98-b0af-455e-ae48-6c402d6da46e None admin] RBAC: Authorizing `identity:get_project(project_id=712b019509924e58b3f25b0aca4d7288)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:02:51.006833 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff955a98-b0af-455e-ae48-6c402d6da46e None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:02:51.008875 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 596/1199] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Jun 2 19:02:50 2022] GET /identity/v3/projects/712b019509924e58b3f25b0aca4d7288 => generated 361 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:02:51.015813 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903a8f22-7fbf-446d-a813-3b63dba8c133 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:02:51.024286 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:02:51.134823 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903a8f22-7fbf-446d-a813-3b63dba8c133 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:02:51.174030 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c623eec6-9b50-4853-b427-b6b0b79af4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:02:51.174692 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c623eec6-9b50-4853-b427-b6b0b79af4c4 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:02:51.175018 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c623eec6-9b50-4853-b427-b6b0b79af4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:02:51.175298 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c623eec6-9b50-4853-b427-b6b0b79af4c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:02:51.176139 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c623eec6-9b50-4853-b427-b6b0b79af4c4 None admin] RBAC: Adding query filter params (role.id=f13fdd0847924014abf1e95594195e78, scope.project.id=712b019509924e58b3f25b0aca4d7288, user.id=257c9459782d4c2ca243f08ed6f163cf) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:02:51.176455 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c623eec6-9b50-4853-b427-b6b0b79af4c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:02:51.177931 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c623eec6-9b50-4853-b427-b6b0b79af4c4 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:02:51.186207 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 604/1200] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 2 19:02:51 2022] GET /identity/v3/role_assignments?user.id=257c9459782d4c2ca243f08ed6f163cf&scope.project.id=712b019509924e58b3f25b0aca4d7288&role.id=f13fdd0847924014abf1e95594195e78 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:03:00.997649 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=51 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:03:01.182610 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=65 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:13.214017 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859a0c6a-9fec-442b-8969-e2b801639db1 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:13.214664 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859a0c6a-9fec-442b-8969-e2b801639db1 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:13.214664 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859a0c6a-9fec-442b-8969-e2b801639db1 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:13.216076 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 597/1201] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:04:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:04:13.221277 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b6339-13ce-43f7-b2c8-12ea6bc9bfc6 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:13.221694 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b6339-13ce-43f7-b2c8-12ea6bc9bfc6 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:13.221835 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b6339-13ce-43f7-b2c8-12ea6bc9bfc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:13.239141 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-204b6339-13ce-43f7-b2c8-12ea6bc9bfc6 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:04:13.240337 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-204b6339-13ce-43f7-b2c8-12ea6bc9bfc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:04:13.310459 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 605/1202] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:04:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 02 19:04:13.335886 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859a0c6a-9fec-442b-8969-e2b801639db1 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:13.439503 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859a0c6a-9fec-442b-8969-e2b801639db1 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:13.474838 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1826004f-b7c9-40ce-9c0b-35890d90ba09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:13.475675 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1826004f-b7c9-40ce-9c0b-35890d90ba09 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:13.475958 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1826004f-b7c9-40ce-9c0b-35890d90ba09 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:13.476229 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1826004f-b7c9-40ce-9c0b-35890d90ba09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:13.487494 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1826004f-b7c9-40ce-9c0b-35890d90ba09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:13.488994 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1826004f-b7c9-40ce-9c0b-35890d90ba09 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:13.507778 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-1826004f-b7c9-40ce-9c0b-35890d90ba09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:04:13.507778 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:04:13.507778 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:04:13.507778 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:04:13.507778 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:04:13.507778 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:04:13.507778 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:04:13.507778 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:04:13.507778 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:04:13.507778 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:04:13.507778 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:04:13.507778 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:04:13.507778 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:04:13.507778 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:04:13.507778 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:04:13.508937 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:04:13.508937 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:04:13.508937 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:04:13.508937 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:04:13.508937 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:04:13.508937 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:04:13.508937 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:04:13.508937 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:04:13.508937 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:04:13.508937 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:04:13.508937 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:04:13.508937 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:04:13.508937 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:04:13.508937 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:04:13.508937 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:04:13.508937 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:04:13.508937 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:04:13.509960 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:04:13.509960 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:04:13.509960 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:04:13.509960 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:04:13.509960 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 598/1203] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:04:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:04:13.515132 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204b6339-13ce-43f7-b2c8-12ea6bc9bfc6 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:13.615630 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204b6339-13ce-43f7-b2c8-12ea6bc9bfc6 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:13.645094 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:13.650924 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0403f23d-7c11-44eb-a99c-e82471b0fb8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:13.651599 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403f23d-7c11-44eb-a99c-e82471b0fb8f None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:13.651883 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403f23d-7c11-44eb-a99c-e82471b0fb8f None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:13.652219 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403f23d-7c11-44eb-a99c-e82471b0fb8f None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:13.652974 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0403f23d-7c11-44eb-a99c-e82471b0fb8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:04:13.653285 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0403f23d-7c11-44eb-a99c-e82471b0fb8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:13.654665 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0403f23d-7c11-44eb-a99c-e82471b0fb8f None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:13.666593 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 606/1204] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:04:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:04:13.672773 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1826004f-b7c9-40ce-9c0b-35890d90ba09 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:13.725739 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:13.773366 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1826004f-b7c9-40ce-9c0b-35890d90ba09 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:13.807235 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc0e9dd-05fb-4ee4-abaa-1439da8721d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:13.807913 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc0e9dd-05fb-4ee4-abaa-1439da8721d0 None admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:13.808201 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc0e9dd-05fb-4ee4-abaa-1439da8721d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:13.808482 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc0e9dd-05fb-4ee4-abaa-1439da8721d0 None admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:13.809182 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc0e9dd-05fb-4ee4-abaa-1439da8721d0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:13.810610 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc0e9dd-05fb-4ee4-abaa-1439da8721d0 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:13.935868 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 599/1205] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Jun 2 19:04:13 2022] POST /identity/v3/users => generated 251 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:04:16.223342 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c74091-d749-40a2-8537-4678bd78329e None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:16.223893 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c74091-d749-40a2-8537-4678bd78329e None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:16.223893 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c74091-d749-40a2-8537-4678bd78329e None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:16.225265 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 607/1206] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:04:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:04:16.230821 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff5de52-6e0b-4ebb-93d0-7e69227038fc None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:16.231273 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff5de52-6e0b-4ebb-93d0-7e69227038fc None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:16.231382 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff5de52-6e0b-4ebb-93d0-7e69227038fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:16.249279 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-bff5de52-6e0b-4ebb-93d0-7e69227038fc None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:04:16.250501 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-bff5de52-6e0b-4ebb-93d0-7e69227038fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:04:16.318570 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 600/1207] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:04:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 02 19:04:16.342666 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c74091-d749-40a2-8537-4678bd78329e None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:16.447052 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c74091-d749-40a2-8537-4678bd78329e None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:16.484352 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cad578d-14ce-4c41-b0e4-5660556a7e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:16.485152 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cad578d-14ce-4c41-b0e4-5660556a7e22 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:16.485435 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cad578d-14ce-4c41-b0e4-5660556a7e22 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:16.485707 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cad578d-14ce-4c41-b0e4-5660556a7e22 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:16.496689 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cad578d-14ce-4c41-b0e4-5660556a7e22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:04:16.506548 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-7cad578d-14ce-4c41-b0e4-5660556a7e22 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 02 19:04:16.506548 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:04:16.506548 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:04:16.506548 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:04:16.506548 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:04:16.506548 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:04:16.506548 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:04:16.506548 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:04:16.506548 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:04:16.506548 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:04:16.506548 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:04:16.506548 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:04:16.506548 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:04:16.506548 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:04:16.506548 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:04:16.506548 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:04:16.507685 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:04:16.507685 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:04:16.507685 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:04:16.507685 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:04:16.507685 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:04:16.507685 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:04:16.507685 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:04:16.507685 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:04:16.507685 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:04:16.507685 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:04:16.507685 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:04:16.507685 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:04:16.507685 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:04:16.507685 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:04:16.507685 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:04:16.507685 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:04:16.509467 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:04:16.509467 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:04:16.509467 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:04:16.509467 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:04:16.509467 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:04:16.509467 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:04:16.509467 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:04:16.509467 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:04:16.509467 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:04:16.509467 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:04:16.509467 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 02 19:04:16.509467 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:04:16.509467 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 608/1208] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 2 19:04:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:04:16.514418 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff5de52-6e0b-4ebb-93d0-7e69227038fc None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:16.557072 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:16.615440 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff5de52-6e0b-4ebb-93d0-7e69227038fc None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:16.649127 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c28f3d6-c83f-4d8a-b440-08426ded88bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:16.649801 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c28f3d6-c83f-4d8a-b440-08426ded88bd None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:16.650084 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c28f3d6-c83f-4d8a-b440-08426ded88bd None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:16.650360 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c28f3d6-c83f-4d8a-b440-08426ded88bd None admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:16.651140 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c28f3d6-c83f-4d8a-b440-08426ded88bd None admin] RBAC: Adding query filter params (name=service) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:04:16.651448 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c28f3d6-c83f-4d8a-b440-08426ded88bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:16.652817 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c28f3d6-c83f-4d8a-b440-08426ded88bd None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:16.660694 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 601/1209] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 2 19:04:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:04:16.667137 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cad578d-14ce-4c41-b0e4-5660556a7e22 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:16.723096 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:16.776796 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cad578d-14ce-4c41-b0e4-5660556a7e22 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:16.813482 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93326cb7-f0a6-4fa4-acfe-e0a1c7993b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:16.814298 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93326cb7-f0a6-4fa4-acfe-e0a1c7993b53 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:16.814586 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93326cb7-f0a6-4fa4-acfe-e0a1c7993b53 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:16.814909 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93326cb7-f0a6-4fa4-acfe-e0a1c7993b53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:16.826564 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93326cb7-f0a6-4fa4-acfe-e0a1c7993b53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:16.828077 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93326cb7-f0a6-4fa4-acfe-e0a1c7993b53 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:16.849097 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-93326cb7-f0a6-4fa4-acfe-e0a1c7993b53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:04:16.849097 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:04:16.849097 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:04:16.849097 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:04:16.849097 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:04:16.849097 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:04:16.849097 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:04:16.849097 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:04:16.849097 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:04:16.849097 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:04:16.849097 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:04:16.849097 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:04:16.849097 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:04:16.849097 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:04:16.849097 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:04:16.850238 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:04:16.850238 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:04:16.850238 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:04:16.850238 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:04:16.850238 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:04:16.850238 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:04:16.850238 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:04:16.850238 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:04:16.850238 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:04:16.850238 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:04:16.850238 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:04:16.850238 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:04:16.850238 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:04:16.850238 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:04:16.850238 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:04:16.850238 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:04:16.850238 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:04:16.865111 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:04:16.865111 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:04:16.865111 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:04:16.865111 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:04:16.865111 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 609/1210] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:04:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:04:16.865487 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c28f3d6-c83f-4d8a-b440-08426ded88bd None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:16.957311 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:16.960294 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c28f3d6-c83f-4d8a-b440-08426ded88bd None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:16.994104 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834578ce-7733-43f2-a626-2638e1e0f0f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:16.994705 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834578ce-7733-43f2-a626-2638e1e0f0f7 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:16.995029 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834578ce-7733-43f2-a626-2638e1e0f0f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:16.995307 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834578ce-7733-43f2-a626-2638e1e0f0f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:16.996039 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834578ce-7733-43f2-a626-2638e1e0f0f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:04:16.996350 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834578ce-7733-43f2-a626-2638e1e0f0f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:16.997713 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834578ce-7733-43f2-a626-2638e1e0f0f7 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:17.009640 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 602/1211] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:04:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:04:17.015293 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93326cb7-f0a6-4fa4-acfe-e0a1c7993b53 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:17.113782 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93326cb7-f0a6-4fa4-acfe-e0a1c7993b53 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:17.115517 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:17.150470 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0545a5ad-c2fe-4d4f-861c-c93d63b381f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:17.151296 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0545a5ad-c2fe-4d4f-861c-c93d63b381f9 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:17.151575 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0545a5ad-c2fe-4d4f-861c-c93d63b381f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:17.151843 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0545a5ad-c2fe-4d4f-861c-c93d63b381f9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:17.158860 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0545a5ad-c2fe-4d4f-861c-c93d63b381f9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:17.160315 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0545a5ad-c2fe-4d4f-861c-c93d63b381f9 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:17.175602 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-0545a5ad-c2fe-4d4f-861c-c93d63b381f9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 02 19:04:17.175602 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:04:17.175602 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:04:17.175602 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:04:17.175602 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:04:17.175602 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:04:17.175602 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:04:17.175602 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:04:17.175602 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:04:17.175602 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:04:17.175602 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:04:17.175602 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:04:17.175602 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 02 19:04:17.175602 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 02 19:04:17.175602 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 02 19:04:17.175602 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 02 19:04:17.180969 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:04:17.180969 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:04:17.180969 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 02 19:04:17.180969 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:04:17.180969 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 02 19:04:17.180969 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:04:17.180969 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:04:17.180969 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:04:17.180969 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:04:17.180969 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:04:17.180969 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:04:17.180969 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:04:17.180969 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:04:17.180969 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:04:17.180969 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:04:17.180969 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:04:17.180969 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:04:17.180969 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:04:17.182058 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:04:17.182058 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:04:17.182058 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 02 19:04:17.182058 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 02 19:04:17.182058 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 02 19:04:17.182058 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 02 19:04:17.182058 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 02 19:04:17.182058 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 02 19:04:17.182058 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 02 19:04:17.182058 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:04:17.182058 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 610/1212] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:04:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:04:17.182967 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834578ce-7733-43f2-a626-2638e1e0f0f7 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:17.286251 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834578ce-7733-43f2-a626-2638e1e0f0f7 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:17.318455 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d11163-24b1-4235-a981-f4dea2b31942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:17.319159 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d11163-24b1-4235-a981-f4dea2b31942 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:17.319446 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d11163-24b1-4235-a981-f4dea2b31942 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:17.319720 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d11163-24b1-4235-a981-f4dea2b31942 None admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:17.320554 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d11163-24b1-4235-a981-f4dea2b31942 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:04:17.320861 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d11163-24b1-4235-a981-f4dea2b31942 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:17.322888 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d11163-24b1-4235-a981-f4dea2b31942 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:17.340399 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 603/1213] 10.222.0.33 () {56 vars in 1143 bytes} [Thu Jun 2 19:04:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:04:17.346091 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0545a5ad-c2fe-4d4f-861c-c93d63b381f9 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:17.447750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0545a5ad-c2fe-4d4f-861c-c93d63b381f9 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:17.483373 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab2ee9e-f63a-4fb2-86a1-ff8b029788f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:17.484030 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab2ee9e-f63a-4fb2-86a1-ff8b029788f5 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:17.484320 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab2ee9e-f63a-4fb2-86a1-ff8b029788f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:17.484598 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab2ee9e-f63a-4fb2-86a1-ff8b029788f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:17.515558 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:17.553128 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab2ee9e-f63a-4fb2-86a1-ff8b029788f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:17.554612 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab2ee9e-f63a-4fb2-86a1-ff8b029788f5 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:17.652204 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 611/1214] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 2 19:04:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 02 19:04:17.658298 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d11163-24b1-4235-a981-f4dea2b31942 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:17.677103 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:17.757744 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d11163-24b1-4235-a981-f4dea2b31942 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:17.793691 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff41270-9aa8-45df-ba2c-e1b5f8210b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:17.794505 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff41270-9aa8-45df-ba2c-e1b5f8210b75 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:17.794821 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff41270-9aa8-45df-ba2c-e1b5f8210b75 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:17.795099 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff41270-9aa8-45df-ba2c-e1b5f8210b75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:17.805221 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff41270-9aa8-45df-ba2c-e1b5f8210b75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:17.807147 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff41270-9aa8-45df-ba2c-e1b5f8210b75 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:17.825622 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-2ff41270-9aa8-45df-ba2c-e1b5f8210b75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:04:17.825622 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:04:17.825622 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:04:17.825622 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:04:17.825622 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:04:17.825622 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:04:17.825622 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:04:17.825622 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:04:17.825622 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:04:17.825622 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:04:17.825622 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:04:17.825622 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:04:17.825622 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:04:17.825622 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:04:17.825622 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:04:17.831852 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:04:17.831852 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:04:17.831852 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:04:17.831852 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:04:17.831852 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:04:17.831852 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:04:17.831852 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:04:17.831852 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:04:17.831852 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:04:17.831852 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:04:17.831852 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:04:17.831852 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:04:17.831852 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:04:17.831852 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:04:17.831852 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:04:17.831852 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:04:17.831852 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:04:17.832867 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:04:17.832867 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:04:17.832867 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:04:17.832867 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:04:17.832867 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 604/1215] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:04:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:04:17.833200 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab2ee9e-f63a-4fb2-86a1-ff8b029788f5 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:17.894877 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:17.932205 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab2ee9e-f63a-4fb2-86a1-ff8b029788f5 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:17.965438 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652ae1b1-16a0-455e-980b-90a21956def7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:17.966100 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652ae1b1-16a0-455e-980b-90a21956def7 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:17.966385 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652ae1b1-16a0-455e-980b-90a21956def7 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:17.966706 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652ae1b1-16a0-455e-980b-90a21956def7 None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:17.967471 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652ae1b1-16a0-455e-980b-90a21956def7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:04:17.967997 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652ae1b1-16a0-455e-980b-90a21956def7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:17.970176 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652ae1b1-16a0-455e-980b-90a21956def7 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:17.982296 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 612/1216] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:04:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:04:17.988238 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff41270-9aa8-45df-ba2c-e1b5f8210b75 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:18.058673 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:18.088085 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff41270-9aa8-45df-ba2c-e1b5f8210b75 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:18.122861 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e520ffd3-c2dd-4738-8e9b-b7f5d7c1c9fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:18.123656 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e520ffd3-c2dd-4738-8e9b-b7f5d7c1c9fd None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:18.123935 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e520ffd3-c2dd-4738-8e9b-b7f5d7c1c9fd None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:18.124206 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e520ffd3-c2dd-4738-8e9b-b7f5d7c1c9fd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:18.131028 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e520ffd3-c2dd-4738-8e9b-b7f5d7c1c9fd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:18.132470 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e520ffd3-c2dd-4738-8e9b-b7f5d7c1c9fd None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:18.146487 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-e520ffd3-c2dd-4738-8e9b-b7f5d7c1c9fd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:04:18.146487 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:04:18.146487 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:04:18.146487 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:04:18.146487 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:04:18.146487 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:04:18.146487 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:04:18.146487 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:04:18.146487 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:04:18.146487 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:04:18.146487 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:04:18.146487 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:04:18.146487 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 02 19:04:18.146487 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 02 19:04:18.146487 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 02 19:04:18.149124 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 02 19:04:18.149124 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:04:18.149124 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:04:18.149124 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:04:18.149124 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:04:18.149124 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:04:18.149124 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:04:18.149124 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:04:18.149124 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:04:18.149124 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:04:18.149124 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:04:18.149124 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:04:18.149124 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:04:18.149124 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:04:18.149124 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:04:18.149124 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:04:18.149124 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:04:18.150837 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 02 19:04:18.150837 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 02 19:04:18.150837 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 02 19:04:18.150837 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 02 19:04:18.150837 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 02 19:04:18.150837 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 02 19:04:18.150837 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:04:18.150837 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:04:18.150837 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 605/1217] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 2 19:04:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:04:18.154567 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652ae1b1-16a0-455e-980b-90a21956def7 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:18.258288 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652ae1b1-16a0-455e-980b-90a21956def7 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:18.279798 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:18.291480 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89e9a73-996b-418f-b125-baa86cc65697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:18.292144 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89e9a73-996b-418f-b125-baa86cc65697 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:18.292431 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89e9a73-996b-418f-b125-baa86cc65697 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:18.292706 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89e9a73-996b-418f-b125-baa86cc65697 None admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:18.293493 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89e9a73-996b-418f-b125-baa86cc65697 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:04:18.293802 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89e9a73-996b-418f-b125-baa86cc65697 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:18.295279 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89e9a73-996b-418f-b125-baa86cc65697 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:18.307762 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 613/1218] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 2 19:04:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:04:18.313985 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e520ffd3-c2dd-4738-8e9b-b7f5d7c1c9fd None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:18.415595 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e520ffd3-c2dd-4738-8e9b-b7f5d7c1c9fd None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:18.444768 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:18.450849 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6f7c67-36d8-4b5d-b36e-93193fb752d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:18.451513 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6f7c67-36d8-4b5d-b36e-93193fb752d0 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:18.451799 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6f7c67-36d8-4b5d-b36e-93193fb752d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:18.452078 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6f7c67-36d8-4b5d-b36e-93193fb752d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:18.452912 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6f7c67-36d8-4b5d-b36e-93193fb752d0 None admin] RBAC: Adding query filter params (role.id=0e56fba112f94ebba363ffab7f4aa9fe, scope.project.id=8c04020eaebd4647804cf8653f32d9e5, user.id=73ae837e99764bebbe7a6e06ac9049e3) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:04:18.453224 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6f7c67-36d8-4b5d-b36e-93193fb752d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:18.454652 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6f7c67-36d8-4b5d-b36e-93193fb752d0 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:18.460065 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 606/1219] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 2 19:04:18 2022] GET /identity/v3/role_assignments?user.id=73ae837e99764bebbe7a6e06ac9049e3&scope.project.id=8c04020eaebd4647804cf8653f32d9e5&role.id=0e56fba112f94ebba363ffab7f4aa9fe => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:04:20.720430 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c55fd2-6f4a-4224-91cb-404dcde67787 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:20.721014 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c55fd2-6f4a-4224-91cb-404dcde67787 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:20.721014 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c55fd2-6f4a-4224-91cb-404dcde67787 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:20.722501 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 614/1220] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:04:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:04:20.729123 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a986bee7-31e5-44b8-a674-00f452108244 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:20.729585 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a986bee7-31e5-44b8-a674-00f452108244 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:20.729699 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a986bee7-31e5-44b8-a674-00f452108244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:20.746416 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-a986bee7-31e5-44b8-a674-00f452108244 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:04:20.749877 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-a986bee7-31e5-44b8-a674-00f452108244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:04:20.817572 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 607/1221] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:04:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 02 19:04:20.841117 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c55fd2-6f4a-4224-91cb-404dcde67787 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:20.943754 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c55fd2-6f4a-4224-91cb-404dcde67787 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:20.978782 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d381c7-0dc0-4e12-afac-abfac33ced71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:20.979528 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d381c7-0dc0-4e12-afac-abfac33ced71 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:20.979849 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d381c7-0dc0-4e12-afac-abfac33ced71 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:20.980132 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d381c7-0dc0-4e12-afac-abfac33ced71 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:20.990191 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d381c7-0dc0-4e12-afac-abfac33ced71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:04:20.998904 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-f4d381c7-0dc0-4e12-afac-abfac33ced71 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 02 19:04:20.998904 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:04:20.998904 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:04:20.998904 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:04:20.998904 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:04:20.998904 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:04:20.998904 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:04:20.998904 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:04:20.998904 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:04:20.998904 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:04:20.998904 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:04:20.998904 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:04:20.998904 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:04:20.998904 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:04:20.998904 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:04:20.998904 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:04:21.006850 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:04:21.006850 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:04:21.006850 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:04:21.006850 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:04:21.006850 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:04:21.006850 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:04:21.006850 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:04:21.006850 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:04:21.006850 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:04:21.006850 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:04:21.006850 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:04:21.006850 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:04:21.006850 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:04:21.006850 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:04:21.006850 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:04:21.006850 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:04:21.007888 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:04:21.007888 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:04:21.007888 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:04:21.007888 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:04:21.007888 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:04:21.007888 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:04:21.007888 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:04:21.007888 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:04:21.007888 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:04:21.007888 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:04:21.007888 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 02 19:04:21.007888 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:04:21.007888 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 615/1222] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 2 19:04:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:04:21.008762 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a986bee7-31e5-44b8-a674-00f452108244 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:21.105737 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a986bee7-31e5-44b8-a674-00f452108244 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:21.133024 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:21.143666 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d07572-9e22-4461-9783-1e6d39beafc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:21.144340 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d07572-9e22-4461-9783-1e6d39beafc3 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:21.144624 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d07572-9e22-4461-9783-1e6d39beafc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:21.144899 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d07572-9e22-4461-9783-1e6d39beafc3 None admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:21.145660 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d07572-9e22-4461-9783-1e6d39beafc3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:04:21.145949 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d07572-9e22-4461-9783-1e6d39beafc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:21.147356 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d07572-9e22-4461-9783-1e6d39beafc3 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:21.155206 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 608/1223] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 2 19:04:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:04:21.161116 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d381c7-0dc0-4e12-afac-abfac33ced71 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:21.206744 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:21.262213 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d381c7-0dc0-4e12-afac-abfac33ced71 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:21.295577 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a03e90d-b482-41cb-a83c-b9a0f45c7b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:21.296371 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a03e90d-b482-41cb-a83c-b9a0f45c7b94 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:21.296655 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a03e90d-b482-41cb-a83c-b9a0f45c7b94 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:21.296923 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a03e90d-b482-41cb-a83c-b9a0f45c7b94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:21.307861 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a03e90d-b482-41cb-a83c-b9a0f45c7b94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:21.309314 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a03e90d-b482-41cb-a83c-b9a0f45c7b94 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:21.326294 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-5a03e90d-b482-41cb-a83c-b9a0f45c7b94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:04:21.326294 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:04:21.326294 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:04:21.326294 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:04:21.326294 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:04:21.326294 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:04:21.326294 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:04:21.326294 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:04:21.326294 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:04:21.326294 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:04:21.326294 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:04:21.326294 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:04:21.326294 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:04:21.326294 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:04:21.326294 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:04:21.327409 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:04:21.327409 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:04:21.327409 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:04:21.327409 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:04:21.327409 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:04:21.327409 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:04:21.327409 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:04:21.327409 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:04:21.327409 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:04:21.327409 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:04:21.327409 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:04:21.327409 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:04:21.327409 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:04:21.327409 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:04:21.327409 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:04:21.327409 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:04:21.327409 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:04:21.328753 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:04:21.328753 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:04:21.328753 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:04:21.328753 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:04:21.328753 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 616/1224] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:04:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:04:21.333346 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d07572-9e22-4461-9783-1e6d39beafc3 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:21.435766 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d07572-9e22-4461-9783-1e6d39beafc3 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:21.468721 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae187be-ca0b-427d-93aa-f12ea4a438ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:21.469381 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae187be-ca0b-427d-93aa-f12ea4a438ea None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:21.469662 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae187be-ca0b-427d-93aa-f12ea4a438ea None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:21.469937 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae187be-ca0b-427d-93aa-f12ea4a438ea None admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:21.470657 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae187be-ca0b-427d-93aa-f12ea4a438ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:04:21.470996 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae187be-ca0b-427d-93aa-f12ea4a438ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:21.472366 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae187be-ca0b-427d-93aa-f12ea4a438ea None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:21.484187 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 609/1225] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:04:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:04:21.489960 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a03e90d-b482-41cb-a83c-b9a0f45c7b94 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:21.590035 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:21.596368 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a03e90d-b482-41cb-a83c-b9a0f45c7b94 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:21.629542 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5084e5-04cb-4290-8cdc-71de9c1dd4cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:21.630617 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5084e5-04cb-4290-8cdc-71de9c1dd4cd None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:21.630935 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5084e5-04cb-4290-8cdc-71de9c1dd4cd None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:21.631216 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5084e5-04cb-4290-8cdc-71de9c1dd4cd None admin] PATH_INFO: `/v3/users/placement` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:21.638645 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5084e5-04cb-4290-8cdc-71de9c1dd4cd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:21.640129 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5084e5-04cb-4290-8cdc-71de9c1dd4cd None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:21.654699 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-2f5084e5-04cb-4290-8cdc-71de9c1dd4cd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 02 19:04:21.654699 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:04:21.654699 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:04:21.654699 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:04:21.654699 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:04:21.654699 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:04:21.654699 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:04:21.654699 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:04:21.654699 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:04:21.654699 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:04:21.654699 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:04:21.654699 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:04:21.654699 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 02 19:04:21.654699 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 02 19:04:21.654699 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 02 19:04:21.654699 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 02 19:04:21.655836 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:04:21.655836 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:04:21.655836 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 02 19:04:21.655836 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:04:21.655836 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 02 19:04:21.655836 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:04:21.655836 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:04:21.655836 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:04:21.655836 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:04:21.655836 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:04:21.655836 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:04:21.655836 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:04:21.655836 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:04:21.655836 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:04:21.655836 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:04:21.655836 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:04:21.655836 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:04:21.655836 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:04:21.657102 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:04:21.657102 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:04:21.657102 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 02 19:04:21.657102 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 02 19:04:21.657102 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 02 19:04:21.657102 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 02 19:04:21.657102 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 02 19:04:21.657102 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 02 19:04:21.657102 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 02 19:04:21.657102 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:04:21.657102 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 617/1226] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:04:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:04:21.661738 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae187be-ca0b-427d-93aa-f12ea4a438ea None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:21.708764 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:21.761547 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae187be-ca0b-427d-93aa-f12ea4a438ea None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:21.794478 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953a41d5-6a62-4236-aece-ced19fead3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:21.795199 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a41d5-6a62-4236-aece-ced19fead3cb None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:21.795481 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a41d5-6a62-4236-aece-ced19fead3cb None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:21.795754 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a41d5-6a62-4236-aece-ced19fead3cb None admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:21.796583 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953a41d5-6a62-4236-aece-ced19fead3cb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:04:21.796931 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953a41d5-6a62-4236-aece-ced19fead3cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:21.798308 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953a41d5-6a62-4236-aece-ced19fead3cb None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:21.814735 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 610/1227] 10.222.0.33 () {56 vars in 1143 bytes} [Thu Jun 2 19:04:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:04:21.820574 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5084e5-04cb-4290-8cdc-71de9c1dd4cd None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:21.922327 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5084e5-04cb-4290-8cdc-71de9c1dd4cd None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:21.955039 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ce4ef5-d127-456b-953d-afee341dcc39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:21.955833 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ce4ef5-d127-456b-953d-afee341dcc39 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:21.956118 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ce4ef5-d127-456b-953d-afee341dcc39 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:21.956395 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ce4ef5-d127-456b-953d-afee341dcc39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:21.966889 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ce4ef5-d127-456b-953d-afee341dcc39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:21.968366 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ce4ef5-d127-456b-953d-afee341dcc39 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:21.985704 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-63ce4ef5-d127-456b-953d-afee341dcc39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:04:21.985704 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:04:21.985704 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:04:21.985704 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:04:21.985704 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:04:21.985704 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:04:21.985704 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:04:21.985704 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:04:21.985704 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:04:21.985704 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:04:21.985704 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:04:21.985704 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:04:21.985704 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:04:21.985704 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:04:21.985704 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:04:21.986669 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:04:21.986669 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:04:21.986669 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:04:21.986669 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:04:21.986669 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:04:21.986669 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:04:21.986669 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:04:21.986669 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:04:21.986669 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:04:21.986669 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:04:21.986669 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:04:21.986669 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:04:21.986669 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:04:21.986669 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:04:21.986669 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:04:21.986669 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:04:21.986669 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:04:21.993287 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:04:21.993287 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:04:21.993287 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:04:21.993287 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:04:21.993287 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 618/1228] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:04:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:04:21.993640 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953a41d5-6a62-4236-aece-ced19fead3cb None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:22.093222 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953a41d5-6a62-4236-aece-ced19fead3cb None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:22.095599 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:22.130461 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d85d03f-ebdb-4e33-81e5-9e0cbe7adaca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:22.131173 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d85d03f-ebdb-4e33-81e5-9e0cbe7adaca None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:22.131460 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d85d03f-ebdb-4e33-81e5-9e0cbe7adaca None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:22.131739 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d85d03f-ebdb-4e33-81e5-9e0cbe7adaca None admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:22.132567 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d85d03f-ebdb-4e33-81e5-9e0cbe7adaca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:04:22.133016 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d85d03f-ebdb-4e33-81e5-9e0cbe7adaca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:22.134421 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d85d03f-ebdb-4e33-81e5-9e0cbe7adaca None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:22.145741 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 611/1229] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:04:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:04:22.151971 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ce4ef5-d127-456b-953d-afee341dcc39 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:22.161508 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:22.256233 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ce4ef5-d127-456b-953d-afee341dcc39 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:22.289866 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ffc6ba-ca8a-4daa-a507-3cd2e995e0c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:22.290657 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ffc6ba-ca8a-4daa-a507-3cd2e995e0c3 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:22.290966 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ffc6ba-ca8a-4daa-a507-3cd2e995e0c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:22.291241 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ffc6ba-ca8a-4daa-a507-3cd2e995e0c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:22.297918 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ffc6ba-ca8a-4daa-a507-3cd2e995e0c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:22.299408 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ffc6ba-ca8a-4daa-a507-3cd2e995e0c3 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:22.313864 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-04ffc6ba-ca8a-4daa-a507-3cd2e995e0c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:04:22.313864 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:04:22.313864 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:04:22.313864 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:04:22.313864 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:04:22.313864 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:04:22.313864 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:04:22.313864 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:04:22.313864 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:04:22.313864 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:04:22.313864 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:04:22.313864 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:04:22.313864 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 02 19:04:22.313864 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 02 19:04:22.313864 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 02 19:04:22.314946 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 02 19:04:22.314946 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:04:22.314946 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:04:22.314946 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:04:22.314946 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:04:22.314946 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:04:22.314946 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:04:22.314946 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:04:22.314946 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:04:22.314946 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:04:22.314946 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:04:22.314946 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:04:22.314946 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:04:22.314946 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:04:22.314946 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:04:22.314946 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:04:22.314946 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:04:22.316150 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 02 19:04:22.316150 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 02 19:04:22.316150 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 02 19:04:22.316150 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 02 19:04:22.316150 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 02 19:04:22.316150 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 02 19:04:22.316150 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:04:22.316150 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:04:22.316150 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 619/1230] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 2 19:04:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:04:22.321505 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d85d03f-ebdb-4e33-81e5-9e0cbe7adaca None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:22.420524 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d85d03f-ebdb-4e33-81e5-9e0cbe7adaca None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:22.452639 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ee9bf7-63a4-4bdc-a6e0-c11509acba13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:22.453299 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ee9bf7-63a4-4bdc-a6e0-c11509acba13 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:22.453584 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ee9bf7-63a4-4bdc-a6e0-c11509acba13 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:22.453856 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ee9bf7-63a4-4bdc-a6e0-c11509acba13 None admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:22.454630 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ee9bf7-63a4-4bdc-a6e0-c11509acba13 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:04:22.454969 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ee9bf7-63a4-4bdc-a6e0-c11509acba13 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:22.456397 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ee9bf7-63a4-4bdc-a6e0-c11509acba13 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:22.469662 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 612/1231] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 2 19:04:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:04:22.475829 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ffc6ba-ca8a-4daa-a507-3cd2e995e0c3 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:22.541696 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:22.576865 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ffc6ba-ca8a-4daa-a507-3cd2e995e0c3 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:22.610177 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f186b0-d519-4ba4-84c1-6c70e98a087a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:22.611138 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f186b0-d519-4ba4-84c1-6c70e98a087a None admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:22.611420 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f186b0-d519-4ba4-84c1-6c70e98a087a None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:22.611695 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f186b0-d519-4ba4-84c1-6c70e98a087a None admin] PATH_INFO: `/v3/projects/8c04020eaebd4647804cf8653f32d9e5/users/73ae837e99764bebbe7a6e06ac9049e3/roles/0e56fba112f94ebba363ffab7f4aa9fe` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:22.627162 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f186b0-d519-4ba4-84c1-6c70e98a087a None admin] RBAC: Authorizing `identity:create_grant(project_id=8c04020eaebd4647804cf8653f32d9e5, user_id=73ae837e99764bebbe7a6e06ac9049e3, role_id=0e56fba112f94ebba363ffab7f4aa9fe)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:22.628631 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f186b0-d519-4ba4-84c1-6c70e98a087a None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:22.700682 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 620/1232] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 2 19:04:22 2022] PUT /identity/v3/projects/8c04020eaebd4647804cf8653f32d9e5/users/73ae837e99764bebbe7a6e06ac9049e3/roles/0e56fba112f94ebba363ffab7f4aa9fe => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 02 19:04:24.947102 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fc6c88-d715-4525-8592-d24eda62ee29 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:24.947767 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fc6c88-d715-4525-8592-d24eda62ee29 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:24.947767 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fc6c88-d715-4525-8592-d24eda62ee29 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:24.949060 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 613/1233] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:04:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:04:24.954319 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fdc7c9-2dcb-46f8-9a92-d68d957394a9 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:24.954821 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fdc7c9-2dcb-46f8-9a92-d68d957394a9 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:24.954990 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fdc7c9-2dcb-46f8-9a92-d68d957394a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:24.971182 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-90fdc7c9-2dcb-46f8-9a92-d68d957394a9 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:04:24.972375 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-90fdc7c9-2dcb-46f8-9a92-d68d957394a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:04:25.039946 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 621/1234] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:04:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 02 19:04:25.064890 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fc6c88-d715-4525-8592-d24eda62ee29 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:25.090838 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:25.169470 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fc6c88-d715-4525-8592-d24eda62ee29 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:25.202943 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3533268d-8156-4607-a23c-dab8a5a880d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:25.203672 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3533268d-8156-4607-a23c-dab8a5a880d5 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:25.203958 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3533268d-8156-4607-a23c-dab8a5a880d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:25.204237 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3533268d-8156-4607-a23c-dab8a5a880d5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:25.214910 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3533268d-8156-4607-a23c-dab8a5a880d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:04:25.223375 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-3533268d-8156-4607-a23c-dab8a5a880d5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 02 19:04:25.223375 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:04:25.223375 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:04:25.223375 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:04:25.223375 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:04:25.223375 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:04:25.223375 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:04:25.223375 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:04:25.223375 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:04:25.223375 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:04:25.223375 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:04:25.223375 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:04:25.223375 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:04:25.223375 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:04:25.223375 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:04:25.223375 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:04:25.224614 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:04:25.224614 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:04:25.224614 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:04:25.224614 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:04:25.224614 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:04:25.224614 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:04:25.224614 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:04:25.224614 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:04:25.224614 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:04:25.224614 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:04:25.224614 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:04:25.224614 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:04:25.224614 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:04:25.224614 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:04:25.224614 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:04:25.224614 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:04:25.225972 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:04:25.225972 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:04:25.225972 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:04:25.225972 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:04:25.225972 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:04:25.225972 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:04:25.225972 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:04:25.225972 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:04:25.225972 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:04:25.225972 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:04:25.225972 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 02 19:04:25.225972 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:04:25.225972 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 614/1235] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Jun 2 19:04:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:04:25.231378 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fdc7c9-2dcb-46f8-9a92-d68d957394a9 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:25.278042 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:25.330042 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fdc7c9-2dcb-46f8-9a92-d68d957394a9 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:25.364043 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b911e1-43a8-4613-b296-5a4c467cef20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:25.364726 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b911e1-43a8-4613-b296-5a4c467cef20 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:25.365012 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b911e1-43a8-4613-b296-5a4c467cef20 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:25.365290 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b911e1-43a8-4613-b296-5a4c467cef20 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:25.366037 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b911e1-43a8-4613-b296-5a4c467cef20 None admin] RBAC: Adding query filter params (name=service) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:04:25.366372 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b911e1-43a8-4613-b296-5a4c467cef20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:25.368307 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b911e1-43a8-4613-b296-5a4c467cef20 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:25.376781 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 622/1236] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Jun 2 19:04:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:04:25.383478 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3533268d-8156-4607-a23c-dab8a5a880d5 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:25.471333 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:25.488445 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3533268d-8156-4607-a23c-dab8a5a880d5 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:25.523279 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b68df96-58fc-4a10-a8a1-ea2b25932f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:25.524648 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68df96-58fc-4a10-a8a1-ea2b25932f3c None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:25.524929 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68df96-58fc-4a10-a8a1-ea2b25932f3c None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:25.525200 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68df96-58fc-4a10-a8a1-ea2b25932f3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:25.535196 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b68df96-58fc-4a10-a8a1-ea2b25932f3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:25.536647 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b68df96-58fc-4a10-a8a1-ea2b25932f3c None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:25.553907 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-0b68df96-58fc-4a10-a8a1-ea2b25932f3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:04:25.553907 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:04:25.553907 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:04:25.553907 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:04:25.553907 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:04:25.553907 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:04:25.553907 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:04:25.553907 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:04:25.553907 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:04:25.553907 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:04:25.553907 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:04:25.553907 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:04:25.553907 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:04:25.553907 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:04:25.553907 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:04:25.555094 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:04:25.555094 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:04:25.555094 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:04:25.555094 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:04:25.555094 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:04:25.555094 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:04:25.555094 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:04:25.555094 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:04:25.555094 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:04:25.555094 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:04:25.555094 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:04:25.555094 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:04:25.555094 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:04:25.555094 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:04:25.555094 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:04:25.555094 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:04:25.555094 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:04:25.556048 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:04:25.556048 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:04:25.556048 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:04:25.556048 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:04:25.556048 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 615/1237] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:04:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:04:25.561041 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b911e1-43a8-4613-b296-5a4c467cef20 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:25.659701 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b911e1-43a8-4613-b296-5a4c467cef20 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:25.668728 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:25.692620 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1713a3f8-4d70-486d-b2f2-e9a13d1f875b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:25.693281 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1713a3f8-4d70-486d-b2f2-e9a13d1f875b None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:25.693566 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1713a3f8-4d70-486d-b2f2-e9a13d1f875b None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:25.693845 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1713a3f8-4d70-486d-b2f2-e9a13d1f875b None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:25.694566 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1713a3f8-4d70-486d-b2f2-e9a13d1f875b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:04:25.694909 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1713a3f8-4d70-486d-b2f2-e9a13d1f875b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:25.696847 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1713a3f8-4d70-486d-b2f2-e9a13d1f875b None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:25.708121 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 623/1238] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:04:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:04:25.714348 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b68df96-58fc-4a10-a8a1-ea2b25932f3c None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:25.815417 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b68df96-58fc-4a10-a8a1-ea2b25932f3c None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:25.850663 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acee4aa-168d-401d-af8e-cfc296aa9f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:25.851489 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acee4aa-168d-401d-af8e-cfc296aa9f3e None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:25.851771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acee4aa-168d-401d-af8e-cfc296aa9f3e None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:25.852043 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acee4aa-168d-401d-af8e-cfc296aa9f3e None admin] PATH_INFO: `/v3/users/placement` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:25.854797 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:25.859403 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acee4aa-168d-401d-af8e-cfc296aa9f3e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:25.860843 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acee4aa-168d-401d-af8e-cfc296aa9f3e None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:25.876012 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-7acee4aa-168d-401d-af8e-cfc296aa9f3e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 02 19:04:25.876012 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:04:25.876012 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:04:25.876012 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:04:25.876012 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:04:25.876012 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:04:25.876012 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:04:25.876012 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:04:25.876012 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:04:25.876012 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:04:25.876012 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:04:25.876012 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:04:25.876012 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 02 19:04:25.876012 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 02 19:04:25.876012 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 02 19:04:25.876012 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 02 19:04:25.877178 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:04:25.877178 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:04:25.877178 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 02 19:04:25.877178 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:04:25.877178 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 02 19:04:25.877178 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:04:25.877178 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:04:25.877178 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:04:25.877178 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:04:25.877178 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:04:25.877178 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:04:25.877178 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:04:25.877178 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:04:25.877178 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:04:25.877178 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:04:25.877178 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:04:25.877178 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:04:25.877178 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:04:25.878286 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:04:25.878286 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:04:25.878286 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 02 19:04:25.878286 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 02 19:04:25.878286 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 02 19:04:25.878286 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 02 19:04:25.878286 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 02 19:04:25.878286 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 02 19:04:25.878286 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 02 19:04:25.878286 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:04:25.878286 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 616/1239] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:04:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:04:25.884071 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1713a3f8-4d70-486d-b2f2-e9a13d1f875b None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:25.983875 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1713a3f8-4d70-486d-b2f2-e9a13d1f875b None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:26.018213 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebf37c7-f742-4d08-88f3-0386d219dcb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:26.018915 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf37c7-f742-4d08-88f3-0386d219dcb8 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:26.019199 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf37c7-f742-4d08-88f3-0386d219dcb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:26.019472 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf37c7-f742-4d08-88f3-0386d219dcb8 None admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:26.020305 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebf37c7-f742-4d08-88f3-0386d219dcb8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:04:26.020678 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebf37c7-f742-4d08-88f3-0386d219dcb8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:26.022094 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebf37c7-f742-4d08-88f3-0386d219dcb8 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:26.040790 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 624/1240] 10.222.0.33 () {56 vars in 1143 bytes} [Thu Jun 2 19:04:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:04:26.046652 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acee4aa-168d-401d-af8e-cfc296aa9f3e None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:26.150105 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acee4aa-168d-401d-af8e-cfc296aa9f3e None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:26.184832 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf32dcd-8647-4a80-ba53-d19a0a0ff9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:26.185494 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf32dcd-8647-4a80-ba53-d19a0a0ff9a1 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:26.185783 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf32dcd-8647-4a80-ba53-d19a0a0ff9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:26.186066 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf32dcd-8647-4a80-ba53-d19a0a0ff9a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:26.251417 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:26.260286 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf32dcd-8647-4a80-ba53-d19a0a0ff9a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:26.261764 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf32dcd-8647-4a80-ba53-d19a0a0ff9a1 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:26.370883 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 617/1241] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 2 19:04:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 02 19:04:26.376776 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebf37c7-f742-4d08-88f3-0386d219dcb8 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:26.408758 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:26.482589 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebf37c7-f742-4d08-88f3-0386d219dcb8 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:26.517364 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81327ce9-3e30-486b-bc1e-847056a5022a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:26.518158 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81327ce9-3e30-486b-bc1e-847056a5022a None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:26.518441 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81327ce9-3e30-486b-bc1e-847056a5022a None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:26.518724 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81327ce9-3e30-486b-bc1e-847056a5022a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:26.529054 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81327ce9-3e30-486b-bc1e-847056a5022a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:26.532789 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81327ce9-3e30-486b-bc1e-847056a5022a None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:26.548515 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-81327ce9-3e30-486b-bc1e-847056a5022a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:04:26.548515 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:04:26.548515 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:04:26.548515 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:04:26.548515 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:04:26.548515 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:04:26.548515 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:04:26.548515 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:04:26.548515 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:04:26.548515 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:04:26.548515 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:04:26.548515 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:04:26.548515 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:04:26.548515 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:04:26.548515 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:04:26.549886 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:04:26.549886 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:04:26.549886 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:04:26.549886 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:04:26.549886 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:04:26.549886 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:04:26.549886 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:04:26.549886 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:04:26.549886 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:04:26.549886 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:04:26.549886 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:04:26.549886 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:04:26.549886 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:04:26.549886 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:04:26.549886 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:04:26.549886 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:04:26.549886 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:04:26.552028 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:04:26.552028 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:04:26.552028 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:04:26.552028 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:04:26.552028 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 625/1242] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:04:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:04:26.557120 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf32dcd-8647-4a80-ba53-d19a0a0ff9a1 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:26.653015 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:26.658882 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf32dcd-8647-4a80-ba53-d19a0a0ff9a1 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:26.692534 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1066c32f-6726-41c2-96a3-90bc780cf0b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:26.693200 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1066c32f-6726-41c2-96a3-90bc780cf0b9 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:26.693487 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1066c32f-6726-41c2-96a3-90bc780cf0b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:26.693766 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1066c32f-6726-41c2-96a3-90bc780cf0b9 None admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:26.694487 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1066c32f-6726-41c2-96a3-90bc780cf0b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:04:26.694829 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1066c32f-6726-41c2-96a3-90bc780cf0b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:26.696787 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1066c32f-6726-41c2-96a3-90bc780cf0b9 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:26.708546 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 618/1243] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:04:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:04:26.715006 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81327ce9-3e30-486b-bc1e-847056a5022a None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:26.795393 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:26.822921 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81327ce9-3e30-486b-bc1e-847056a5022a None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:26.857825 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae988c3-8a11-4f0c-8e48-6efecbda5212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:26.858665 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae988c3-8a11-4f0c-8e48-6efecbda5212 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:26.858985 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae988c3-8a11-4f0c-8e48-6efecbda5212 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:26.859266 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae988c3-8a11-4f0c-8e48-6efecbda5212 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:26.867270 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae988c3-8a11-4f0c-8e48-6efecbda5212 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:26.868762 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae988c3-8a11-4f0c-8e48-6efecbda5212 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:26.883527 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-3ae988c3-8a11-4f0c-8e48-6efecbda5212 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:04:26.883527 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:04:26.883527 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:04:26.883527 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:04:26.883527 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:04:26.883527 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:04:26.883527 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:04:26.883527 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:04:26.883527 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:04:26.883527 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:04:26.883527 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:04:26.883527 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:04:26.883527 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 02 19:04:26.883527 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 02 19:04:26.883527 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 02 19:04:26.884748 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 02 19:04:26.884748 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:04:26.884748 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:04:26.884748 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:04:26.884748 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:04:26.884748 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:04:26.884748 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:04:26.884748 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:04:26.884748 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:04:26.884748 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:04:26.884748 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:04:26.884748 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:04:26.884748 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:04:26.884748 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:04:26.884748 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:04:26.884748 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:04:26.884748 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:04:26.885751 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 02 19:04:26.885751 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 02 19:04:26.885751 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 02 19:04:26.885751 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 02 19:04:26.885751 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 02 19:04:26.885751 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 02 19:04:26.885751 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:04:26.885751 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:04:26.885751 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 626/1244] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 2 19:04:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:04:26.890843 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1066c32f-6726-41c2-96a3-90bc780cf0b9 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:26.991157 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1066c32f-6726-41c2-96a3-90bc780cf0b9 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:27.025796 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2572bfc9-f398-492a-81ee-bea6408e3be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:27.026462 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2572bfc9-f398-492a-81ee-bea6408e3be2 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:27.026776 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2572bfc9-f398-492a-81ee-bea6408e3be2 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:27.027057 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2572bfc9-f398-492a-81ee-bea6408e3be2 None admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:27.027852 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2572bfc9-f398-492a-81ee-bea6408e3be2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:04:27.028165 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2572bfc9-f398-492a-81ee-bea6408e3be2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:27.029608 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2572bfc9-f398-492a-81ee-bea6408e3be2 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:27.043338 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 619/1245] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 2 19:04:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:04:27.049785 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae988c3-8a11-4f0c-8e48-6efecbda5212 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:27.159502 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae988c3-8a11-4f0c-8e48-6efecbda5212 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:27.196080 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73bd7e7-54bc-46dc-8fcb-7fb73c9f2aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:27.196746 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73bd7e7-54bc-46dc-8fcb-7fb73c9f2aee None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:27.197031 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73bd7e7-54bc-46dc-8fcb-7fb73c9f2aee None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:27.197309 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73bd7e7-54bc-46dc-8fcb-7fb73c9f2aee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:27.198153 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73bd7e7-54bc-46dc-8fcb-7fb73c9f2aee None admin] RBAC: Adding query filter params (role.id=0e56fba112f94ebba363ffab7f4aa9fe, scope.project.id=8c04020eaebd4647804cf8653f32d9e5, user.id=73ae837e99764bebbe7a6e06ac9049e3) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:04:27.198468 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73bd7e7-54bc-46dc-8fcb-7fb73c9f2aee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:27.200008 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73bd7e7-54bc-46dc-8fcb-7fb73c9f2aee None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:27.206776 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 627/1246] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 2 19:04:27 2022] GET /identity/v3/role_assignments?user.id=73ae837e99764bebbe7a6e06ac9049e3&scope.project.id=8c04020eaebd4647804cf8653f32d9e5&role.id=0e56fba112f94ebba363ffab7f4aa9fe => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:04:29.503359 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8032a707-5d3c-457f-86d8-c37ba2e0e7d3 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:29.504192 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8032a707-5d3c-457f-86d8-c37ba2e0e7d3 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:29.504192 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8032a707-5d3c-457f-86d8-c37ba2e0e7d3 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:29.505568 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 620/1247] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:04:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:04:29.511136 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c378cb42-d2ed-4ba8-b750-0600d31b970d None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:29.511622 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c378cb42-d2ed-4ba8-b750-0600d31b970d None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:29.511753 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c378cb42-d2ed-4ba8-b750-0600d31b970d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:29.516896 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:29.529459 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-c378cb42-d2ed-4ba8-b750-0600d31b970d None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:04:29.530693 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-c378cb42-d2ed-4ba8-b750-0600d31b970d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:04:29.600095 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 628/1248] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:04:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 02 19:04:29.625507 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8032a707-5d3c-457f-86d8-c37ba2e0e7d3 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:29.643867 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:29.727123 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8032a707-5d3c-457f-86d8-c37ba2e0e7d3 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:29.763356 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cd1197-cdd6-4c9e-9b3a-4522868b1f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:29.764073 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd1197-cdd6-4c9e-9b3a-4522868b1f1a None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:29.764359 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd1197-cdd6-4c9e-9b3a-4522868b1f1a None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:29.764637 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd1197-cdd6-4c9e-9b3a-4522868b1f1a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:29.775427 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cd1197-cdd6-4c9e-9b3a-4522868b1f1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:04:29.783180 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-e1cd1197-cdd6-4c9e-9b3a-4522868b1f1a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 02 19:04:29.783180 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:04:29.783180 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:04:29.783180 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:04:29.783180 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:04:29.783180 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:04:29.783180 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:04:29.783180 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:04:29.783180 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:04:29.783180 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:04:29.783180 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:04:29.783180 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:04:29.783180 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:04:29.783180 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:04:29.783180 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:04:29.783180 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:04:29.784226 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:04:29.784226 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:04:29.784226 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:04:29.784226 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:04:29.784226 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:04:29.784226 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:04:29.784226 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:04:29.784226 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:04:29.784226 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:04:29.784226 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:04:29.784226 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:04:29.784226 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:04:29.784226 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:04:29.784226 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:04:29.784226 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:04:29.784226 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:04:29.785060 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:04:29.785060 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:04:29.785060 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:04:29.785060 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:04:29.785060 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:04:29.785060 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:04:29.785060 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:04:29.785060 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:04:29.785060 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:04:29.785060 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:04:29.785060 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 02 19:04:29.785060 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:04:29.785060 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 621/1249] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 2 19:04:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:04:29.790505 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c378cb42-d2ed-4ba8-b750-0600d31b970d None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:29.891070 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c378cb42-d2ed-4ba8-b750-0600d31b970d None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:29.920626 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:29.925489 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9f44f9-bfcd-45fb-9220-da982c712b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:29.926164 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f44f9-bfcd-45fb-9220-da982c712b53 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:29.926451 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f44f9-bfcd-45fb-9220-da982c712b53 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:29.926728 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f44f9-bfcd-45fb-9220-da982c712b53 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:29.927578 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9f44f9-bfcd-45fb-9220-da982c712b53 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:04:29.927887 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9f44f9-bfcd-45fb-9220-da982c712b53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:29.929795 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9f44f9-bfcd-45fb-9220-da982c712b53 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:29.951578 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 629/1250] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 2 19:04:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:04:29.961392 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cd1197-cdd6-4c9e-9b3a-4522868b1f1a None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:30.050031 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:30.074059 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cd1197-cdd6-4c9e-9b3a-4522868b1f1a None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:30.109208 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadbecd5-ddec-4521-9540-0b467ff90ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:30.110064 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadbecd5-ddec-4521-9540-0b467ff90ca9 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:30.110351 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadbecd5-ddec-4521-9540-0b467ff90ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:30.110629 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadbecd5-ddec-4521-9540-0b467ff90ca9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:30.123283 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadbecd5-ddec-4521-9540-0b467ff90ca9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:30.124778 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadbecd5-ddec-4521-9540-0b467ff90ca9 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:30.144494 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-aadbecd5-ddec-4521-9540-0b467ff90ca9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:04:30.144494 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:04:30.144494 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:04:30.144494 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:04:30.144494 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:04:30.144494 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:04:30.144494 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:04:30.144494 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:04:30.144494 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:04:30.144494 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:04:30.144494 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:04:30.144494 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:04:30.144494 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:04:30.144494 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:04:30.144494 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:04:30.148599 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:04:30.148599 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:04:30.148599 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:04:30.148599 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:04:30.148599 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:04:30.148599 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:04:30.148599 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:04:30.148599 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:04:30.148599 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:04:30.148599 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:04:30.148599 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:04:30.148599 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:04:30.148599 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:04:30.148599 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:04:30.148599 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:04:30.148599 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:04:30.148599 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:04:30.149501 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:04:30.149501 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:04:30.149501 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:04:30.149501 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:04:30.149501 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 622/1251] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:04:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:04:30.152364 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9f44f9-bfcd-45fb-9220-da982c712b53 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:30.263069 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9f44f9-bfcd-45fb-9220-da982c712b53 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:30.300225 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268aa21b-45de-4985-af14-6deccb52cad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:30.300862 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268aa21b-45de-4985-af14-6deccb52cad6 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:30.301153 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268aa21b-45de-4985-af14-6deccb52cad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:30.301434 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268aa21b-45de-4985-af14-6deccb52cad6 None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:30.302166 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268aa21b-45de-4985-af14-6deccb52cad6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:04:30.302481 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268aa21b-45de-4985-af14-6deccb52cad6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:30.303900 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268aa21b-45de-4985-af14-6deccb52cad6 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:30.321432 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 630/1252] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:04:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:04:30.327810 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadbecd5-ddec-4521-9540-0b467ff90ca9 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:30.427845 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadbecd5-ddec-4521-9540-0b467ff90ca9 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:30.453490 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:30.461992 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d3f39a-e56b-4414-adb0-77530df50e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:30.462824 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3f39a-e56b-4414-adb0-77530df50e5a None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:30.463110 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3f39a-e56b-4414-adb0-77530df50e5a None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:30.463384 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3f39a-e56b-4414-adb0-77530df50e5a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:30.470999 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d3f39a-e56b-4414-adb0-77530df50e5a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:30.472431 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d3f39a-e56b-4414-adb0-77530df50e5a None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:30.487553 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-b1d3f39a-e56b-4414-adb0-77530df50e5a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 02 19:04:30.487553 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:04:30.487553 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:04:30.487553 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:04:30.487553 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:04:30.487553 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:04:30.487553 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:04:30.487553 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:04:30.487553 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:04:30.487553 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:04:30.487553 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:04:30.487553 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:04:30.487553 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 02 19:04:30.487553 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 02 19:04:30.487553 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 02 19:04:30.487553 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 02 19:04:30.488582 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:04:30.488582 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:04:30.488582 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 02 19:04:30.488582 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:04:30.488582 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 02 19:04:30.488582 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:04:30.488582 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:04:30.488582 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:04:30.488582 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:04:30.488582 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:04:30.488582 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:04:30.488582 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:04:30.488582 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:04:30.488582 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:04:30.488582 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:04:30.488582 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:04:30.488582 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:04:30.488582 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:04:30.489543 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:04:30.489543 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:04:30.489543 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 02 19:04:30.489543 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 02 19:04:30.489543 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 02 19:04:30.489543 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 02 19:04:30.489543 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 02 19:04:30.489543 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 02 19:04:30.489543 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 02 19:04:30.489543 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:04:30.489543 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 623/1253] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:04:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:04:30.495358 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268aa21b-45de-4985-af14-6deccb52cad6 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:30.552544 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:30.597398 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268aa21b-45de-4985-af14-6deccb52cad6 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:30.632879 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8315488a-9d5b-4170-8f6a-48d49f2bafeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:30.633563 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8315488a-9d5b-4170-8f6a-48d49f2bafeb None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:30.633846 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8315488a-9d5b-4170-8f6a-48d49f2bafeb None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:30.634126 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8315488a-9d5b-4170-8f6a-48d49f2bafeb None admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:30.634991 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8315488a-9d5b-4170-8f6a-48d49f2bafeb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:04:30.635308 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8315488a-9d5b-4170-8f6a-48d49f2bafeb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:30.636751 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8315488a-9d5b-4170-8f6a-48d49f2bafeb None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:30.654387 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 631/1254] 10.222.0.33 () {56 vars in 1143 bytes} [Thu Jun 2 19:04:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:04:30.660181 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d3f39a-e56b-4414-adb0-77530df50e5a None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:30.760729 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d3f39a-e56b-4414-adb0-77530df50e5a None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:30.794211 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53db4bf-1c1b-4a7e-baef-4d1c087d8e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:30.794893 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53db4bf-1c1b-4a7e-baef-4d1c087d8e96 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:30.795182 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53db4bf-1c1b-4a7e-baef-4d1c087d8e96 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:30.795459 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53db4bf-1c1b-4a7e-baef-4d1c087d8e96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:30.854381 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:30.866239 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53db4bf-1c1b-4a7e-baef-4d1c087d8e96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:30.867947 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53db4bf-1c1b-4a7e-baef-4d1c087d8e96 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:30.967710 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 624/1255] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 2 19:04:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 02 19:04:30.973588 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8315488a-9d5b-4170-8f6a-48d49f2bafeb None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:31.071655 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8315488a-9d5b-4170-8f6a-48d49f2bafeb None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:31.084463 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:31.105239 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a37e3f-7f95-4cb2-996f-475877027280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:31.105965 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a37e3f-7f95-4cb2-996f-475877027280 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:31.106250 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a37e3f-7f95-4cb2-996f-475877027280 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:31.106527 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a37e3f-7f95-4cb2-996f-475877027280 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:31.117949 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a37e3f-7f95-4cb2-996f-475877027280 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:31.119446 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a37e3f-7f95-4cb2-996f-475877027280 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:31.137778 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-20a37e3f-7f95-4cb2-996f-475877027280 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:04:31.137778 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:04:31.137778 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:04:31.137778 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:04:31.137778 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:04:31.137778 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:04:31.137778 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:04:31.137778 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:04:31.137778 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:04:31.137778 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:04:31.137778 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:04:31.137778 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:04:31.137778 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:04:31.137778 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:04:31.137778 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:04:31.138919 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:04:31.138919 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:04:31.138919 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:04:31.138919 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:04:31.138919 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:04:31.138919 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:04:31.138919 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:04:31.138919 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:04:31.138919 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:04:31.138919 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:04:31.138919 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:04:31.138919 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:04:31.138919 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:04:31.138919 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:04:31.138919 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:04:31.138919 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:04:31.138919 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:04:31.139833 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:04:31.139833 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:04:31.139833 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:04:31.139833 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:04:31.139833 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 632/1256] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:04:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:04:31.144796 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53db4bf-1c1b-4a7e-baef-4d1c087d8e96 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:31.234042 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:31.246582 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53db4bf-1c1b-4a7e-baef-4d1c087d8e96 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:31.282603 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142666b2-90b0-4660-9af9-c2eb92a6af0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:31.283297 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142666b2-90b0-4660-9af9-c2eb92a6af0d None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:31.283585 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142666b2-90b0-4660-9af9-c2eb92a6af0d None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:31.283863 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142666b2-90b0-4660-9af9-c2eb92a6af0d None admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:31.284593 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142666b2-90b0-4660-9af9-c2eb92a6af0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:04:31.284901 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142666b2-90b0-4660-9af9-c2eb92a6af0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:31.286272 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142666b2-90b0-4660-9af9-c2eb92a6af0d None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:31.298291 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 625/1257] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:04:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:04:31.303968 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a37e3f-7f95-4cb2-996f-475877027280 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:31.402608 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a37e3f-7f95-4cb2-996f-475877027280 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:31.435980 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40178954-5a2d-452f-9cbd-ef0e24eb9c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:31.436765 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40178954-5a2d-452f-9cbd-ef0e24eb9c40 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:31.437046 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40178954-5a2d-452f-9cbd-ef0e24eb9c40 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:31.437319 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40178954-5a2d-452f-9cbd-ef0e24eb9c40 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:31.444099 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40178954-5a2d-452f-9cbd-ef0e24eb9c40 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:31.445575 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40178954-5a2d-452f-9cbd-ef0e24eb9c40 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:31.459410 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-40178954-5a2d-452f-9cbd-ef0e24eb9c40 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:04:31.459410 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:04:31.459410 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:04:31.459410 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:04:31.459410 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:04:31.459410 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:04:31.459410 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:04:31.459410 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:04:31.459410 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:04:31.459410 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:04:31.459410 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:04:31.459410 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:04:31.459410 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 02 19:04:31.459410 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 02 19:04:31.459410 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 02 19:04:31.461031 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 02 19:04:31.461031 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:04:31.461031 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:04:31.461031 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:04:31.461031 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:04:31.461031 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:04:31.461031 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:04:31.461031 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:04:31.461031 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:04:31.461031 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:04:31.461031 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:04:31.461031 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:04:31.461031 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:04:31.461031 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:04:31.461031 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:04:31.461031 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:04:31.461031 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:04:31.462115 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 02 19:04:31.462115 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 02 19:04:31.462115 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 02 19:04:31.462115 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 02 19:04:31.462115 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 02 19:04:31.462115 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 02 19:04:31.462115 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:04:31.462115 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:04:31.462115 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 633/1258] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 2 19:04:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:04:31.468819 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142666b2-90b0-4660-9af9-c2eb92a6af0d None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:31.567574 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142666b2-90b0-4660-9af9-c2eb92a6af0d None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:31.603046 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc690ee8-05eb-4c2e-9a87-da5b258929f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:31.603764 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc690ee8-05eb-4c2e-9a87-da5b258929f9 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:31.604054 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc690ee8-05eb-4c2e-9a87-da5b258929f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:31.604337 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc690ee8-05eb-4c2e-9a87-da5b258929f9 None admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:31.605131 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc690ee8-05eb-4c2e-9a87-da5b258929f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:04:31.605455 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc690ee8-05eb-4c2e-9a87-da5b258929f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:31.606928 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc690ee8-05eb-4c2e-9a87-da5b258929f9 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:31.620164 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 626/1259] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 2 19:04:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:04:31.626319 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40178954-5a2d-452f-9cbd-ef0e24eb9c40 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:31.654287 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:31.727298 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40178954-5a2d-452f-9cbd-ef0e24eb9c40 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:31.761396 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1326de21-3b6a-4e2f-b7cb-2cf98342b68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:31.762062 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1326de21-3b6a-4e2f-b7cb-2cf98342b68f None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:31.762348 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1326de21-3b6a-4e2f-b7cb-2cf98342b68f None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:31.762624 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1326de21-3b6a-4e2f-b7cb-2cf98342b68f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:31.763497 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1326de21-3b6a-4e2f-b7cb-2cf98342b68f None admin] RBAC: Adding query filter params (role.id=f13fdd0847924014abf1e95594195e78, scope.project.id=8c04020eaebd4647804cf8653f32d9e5, user.id=73ae837e99764bebbe7a6e06ac9049e3) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:04:31.763809 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1326de21-3b6a-4e2f-b7cb-2cf98342b68f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:31.765262 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1326de21-3b6a-4e2f-b7cb-2cf98342b68f None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:31.770965 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 634/1260] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 2 19:04:31 2022] GET /identity/v3/role_assignments?user.id=73ae837e99764bebbe7a6e06ac9049e3&scope.project.id=8c04020eaebd4647804cf8653f32d9e5&role.id=f13fdd0847924014abf1e95594195e78 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:04:33.996529 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3bba2-1681-4a1f-a8e0-c398c40ab5ea None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:33.997088 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3bba2-1681-4a1f-a8e0-c398c40ab5ea None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:33.997088 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3bba2-1681-4a1f-a8e0-c398c40ab5ea None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:33.998497 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 627/1261] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:04:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:04:34.004282 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f7e8a-a8c0-4229-9a8a-327bd4f6c2c8 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:34.004691 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f7e8a-a8c0-4229-9a8a-327bd4f6c2c8 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:34.008549 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f7e8a-a8c0-4229-9a8a-327bd4f6c2c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:34.021962 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-705f7e8a-a8c0-4229-9a8a-327bd4f6c2c8 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:04:34.023780 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-705f7e8a-a8c0-4229-9a8a-327bd4f6c2c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:04:34.089603 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 635/1262] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:04:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 02 19:04:34.112868 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c3bba2-1681-4a1f-a8e0-c398c40ab5ea None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:34.125018 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:34.213001 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c3bba2-1681-4a1f-a8e0-c398c40ab5ea None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:34.246233 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334561d1-9f13-4bde-8950-326b6f9f2a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:34.247103 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334561d1-9f13-4bde-8950-326b6f9f2a9d None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:34.247333 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334561d1-9f13-4bde-8950-326b6f9f2a9d None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:34.247611 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334561d1-9f13-4bde-8950-326b6f9f2a9d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:34.257894 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334561d1-9f13-4bde-8950-326b6f9f2a9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:04:34.265685 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-334561d1-9f13-4bde-8950-326b6f9f2a9d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 02 19:04:34.265685 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:04:34.265685 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:04:34.265685 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:04:34.265685 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:04:34.265685 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:04:34.265685 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:04:34.265685 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:04:34.265685 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:04:34.265685 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:04:34.265685 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:04:34.265685 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:04:34.265685 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:04:34.265685 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:04:34.265685 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:04:34.265685 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:04:34.266621 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:04:34.266621 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:04:34.266621 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:04:34.266621 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:04:34.266621 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:04:34.266621 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:04:34.266621 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:04:34.266621 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:04:34.266621 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:04:34.266621 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:04:34.266621 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:04:34.266621 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:04:34.266621 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:04:34.266621 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:04:34.266621 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:04:34.266621 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:04:34.267500 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:04:34.267500 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:04:34.267500 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:04:34.267500 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:04:34.267500 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:04:34.267500 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:04:34.267500 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:04:34.267500 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:04:34.267500 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:04:34.267500 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:04:34.267500 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 02 19:04:34.267500 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:04:34.267500 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 628/1263] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 2 19:04:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:04:34.273480 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705f7e8a-a8c0-4229-9a8a-327bd4f6c2c8 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:34.291832 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:34.370440 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705f7e8a-a8c0-4229-9a8a-327bd4f6c2c8 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:34.404110 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950336ba-c84e-48cf-8ddf-306a270983d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:34.404780 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950336ba-c84e-48cf-8ddf-306a270983d4 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:34.405076 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950336ba-c84e-48cf-8ddf-306a270983d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:34.405344 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950336ba-c84e-48cf-8ddf-306a270983d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:34.406094 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950336ba-c84e-48cf-8ddf-306a270983d4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:04:34.406405 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950336ba-c84e-48cf-8ddf-306a270983d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:34.407814 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950336ba-c84e-48cf-8ddf-306a270983d4 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:34.417132 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 636/1264] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 2 19:04:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:04:34.423234 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334561d1-9f13-4bde-8950-326b6f9f2a9d None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:34.490796 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:34.525847 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334561d1-9f13-4bde-8950-326b6f9f2a9d None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:34.558765 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d336d41-323e-4f0b-a29f-2405038858af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:34.559590 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d336d41-323e-4f0b-a29f-2405038858af None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:34.560229 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d336d41-323e-4f0b-a29f-2405038858af None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:34.560710 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d336d41-323e-4f0b-a29f-2405038858af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:34.699700 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d336d41-323e-4f0b-a29f-2405038858af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:34.701244 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d336d41-323e-4f0b-a29f-2405038858af None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:34.720191 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-5d336d41-323e-4f0b-a29f-2405038858af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:04:34.720191 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:04:34.720191 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:04:34.720191 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:04:34.720191 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:04:34.720191 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:04:34.720191 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:04:34.720191 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:04:34.720191 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:04:34.720191 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:04:34.720191 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:04:34.720191 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:04:34.720191 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:04:34.720191 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:04:34.720191 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:04:34.721355 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:04:34.721355 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:04:34.721355 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:04:34.721355 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:04:34.721355 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:04:34.721355 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:04:34.721355 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:04:34.721355 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:04:34.721355 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:04:34.721355 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:04:34.721355 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:04:34.721355 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:04:34.721355 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:04:34.721355 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:04:34.721355 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:04:34.721355 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:04:34.721355 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:04:34.722279 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:04:34.722279 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:04:34.722279 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:04:34.722279 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:04:34.722279 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 629/1265] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:04:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:04:34.728172 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950336ba-c84e-48cf-8ddf-306a270983d4 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:34.811286 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:34.829497 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950336ba-c84e-48cf-8ddf-306a270983d4 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:34.862795 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04664e60-3476-4037-9b05-89b130c4e6d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:34.863786 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04664e60-3476-4037-9b05-89b130c4e6d9 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:34.864069 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04664e60-3476-4037-9b05-89b130c4e6d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:34.864346 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04664e60-3476-4037-9b05-89b130c4e6d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:34.865069 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04664e60-3476-4037-9b05-89b130c4e6d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:04:34.865375 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04664e60-3476-4037-9b05-89b130c4e6d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:34.867404 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04664e60-3476-4037-9b05-89b130c4e6d9 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:34.879602 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 637/1266] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:04:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:04:34.885441 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d336d41-323e-4f0b-a29f-2405038858af None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:34.984611 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d336d41-323e-4f0b-a29f-2405038858af None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:35.005287 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:35.019225 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac6803d-33f0-4436-a41e-92355c40b182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:35.020036 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac6803d-33f0-4436-a41e-92355c40b182 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:35.020318 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac6803d-33f0-4436-a41e-92355c40b182 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:35.020595 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac6803d-33f0-4436-a41e-92355c40b182 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:35.027682 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac6803d-33f0-4436-a41e-92355c40b182 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:35.035227 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac6803d-33f0-4436-a41e-92355c40b182 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:35.043620 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-8ac6803d-33f0-4436-a41e-92355c40b182 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 02 19:04:35.043620 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:04:35.043620 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:04:35.043620 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:04:35.043620 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:04:35.043620 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:04:35.043620 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:04:35.043620 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:04:35.043620 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:04:35.043620 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:04:35.043620 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:04:35.043620 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:04:35.043620 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 02 19:04:35.043620 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 02 19:04:35.043620 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 02 19:04:35.043620 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 02 19:04:35.044859 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:04:35.044859 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:04:35.044859 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 02 19:04:35.044859 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:04:35.044859 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 02 19:04:35.044859 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:04:35.044859 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:04:35.044859 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:04:35.044859 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:04:35.044859 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:04:35.044859 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:04:35.044859 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:04:35.044859 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:04:35.044859 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:04:35.044859 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:04:35.044859 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:04:35.044859 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:04:35.044859 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:04:35.046025 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:04:35.046025 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:04:35.046025 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 02 19:04:35.046025 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 02 19:04:35.046025 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 02 19:04:35.046025 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 02 19:04:35.046025 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 02 19:04:35.046025 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 02 19:04:35.046025 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 02 19:04:35.046025 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:04:35.046025 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 630/1267] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:04:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:04:35.050989 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04664e60-3476-4037-9b05-89b130c4e6d9 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:35.161080 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04664e60-3476-4037-9b05-89b130c4e6d9 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:35.195470 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc3df26-92b2-4c7d-9582-0b1dbdff8f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:35.196220 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3df26-92b2-4c7d-9582-0b1dbdff8f08 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:35.196411 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3df26-92b2-4c7d-9582-0b1dbdff8f08 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:35.196682 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3df26-92b2-4c7d-9582-0b1dbdff8f08 None admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:35.197515 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc3df26-92b2-4c7d-9582-0b1dbdff8f08 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:04:35.197816 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc3df26-92b2-4c7d-9582-0b1dbdff8f08 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:35.199279 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc3df26-92b2-4c7d-9582-0b1dbdff8f08 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:35.208707 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:35.217954 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 638/1268] 10.222.0.33 () {56 vars in 1143 bytes} [Thu Jun 2 19:04:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:04:35.223768 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac6803d-33f0-4436-a41e-92355c40b182 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:35.327794 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac6803d-33f0-4436-a41e-92355c40b182 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:35.361481 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f94df7-d9a3-4845-afa1-2710fb844602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:35.362271 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f94df7-d9a3-4845-afa1-2710fb844602 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:35.362561 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f94df7-d9a3-4845-afa1-2710fb844602 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:35.363006 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f94df7-d9a3-4845-afa1-2710fb844602 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:35.373514 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f94df7-d9a3-4845-afa1-2710fb844602 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:35.374921 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f94df7-d9a3-4845-afa1-2710fb844602 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:35.392554 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-96f94df7-d9a3-4845-afa1-2710fb844602 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:04:35.392554 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:04:35.392554 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:04:35.392554 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:04:35.392554 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:04:35.392554 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:04:35.392554 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:04:35.392554 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:04:35.392554 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:04:35.392554 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:04:35.392554 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:04:35.392554 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:04:35.392554 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:04:35.392554 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:04:35.392554 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:04:35.393611 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:04:35.393611 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:04:35.393611 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:04:35.393611 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:04:35.393611 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:04:35.393611 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:04:35.393611 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:04:35.393611 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:04:35.393611 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:04:35.393611 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:04:35.393611 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:04:35.393611 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:04:35.393611 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:04:35.393611 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:04:35.393611 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:04:35.393611 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:04:35.393611 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:04:35.394782 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:04:35.394782 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:04:35.394782 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:04:35.394782 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:04:35.394782 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 631/1269] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:04:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:04:35.400298 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc3df26-92b2-4c7d-9582-0b1dbdff8f08 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:35.498896 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc3df26-92b2-4c7d-9582-0b1dbdff8f08 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:35.532782 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91436895-2be6-4279-b086-5f661380019d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:35.533437 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91436895-2be6-4279-b086-5f661380019d None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:35.533721 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91436895-2be6-4279-b086-5f661380019d None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:35.533996 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91436895-2be6-4279-b086-5f661380019d None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:35.534714 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91436895-2be6-4279-b086-5f661380019d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:04:35.535055 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91436895-2be6-4279-b086-5f661380019d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:35.536420 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91436895-2be6-4279-b086-5f661380019d None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:35.548441 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 639/1270] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:04:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:04:35.554735 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f94df7-d9a3-4845-afa1-2710fb844602 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:35.585798 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:35.653967 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f94df7-d9a3-4845-afa1-2710fb844602 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:35.689952 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a45d4-e5fb-41dd-9ec7-c411daf98296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:35.690988 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a45d4-e5fb-41dd-9ec7-c411daf98296 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:35.691273 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a45d4-e5fb-41dd-9ec7-c411daf98296 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:35.691552 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a45d4-e5fb-41dd-9ec7-c411daf98296 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:35.699406 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1a45d4-e5fb-41dd-9ec7-c411daf98296 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:35.700885 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1a45d4-e5fb-41dd-9ec7-c411daf98296 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:35.717395 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-1f1a45d4-e5fb-41dd-9ec7-c411daf98296 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:04:35.717395 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:04:35.717395 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:04:35.717395 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:04:35.717395 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:04:35.717395 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:04:35.717395 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:04:35.717395 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:04:35.717395 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:04:35.717395 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:04:35.717395 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:04:35.717395 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:04:35.717395 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 02 19:04:35.717395 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 02 19:04:35.717395 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 02 19:04:35.718537 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 02 19:04:35.718537 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:04:35.718537 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:04:35.718537 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:04:35.718537 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:04:35.718537 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:04:35.718537 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:04:35.718537 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:04:35.718537 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:04:35.718537 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:04:35.718537 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:04:35.718537 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:04:35.718537 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:04:35.718537 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:04:35.718537 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:04:35.718537 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:04:35.718537 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:04:35.719507 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 02 19:04:35.719507 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 02 19:04:35.719507 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 02 19:04:35.719507 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 02 19:04:35.719507 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 02 19:04:35.719507 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 02 19:04:35.719507 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:04:35.719507 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:04:35.719507 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 632/1271] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 2 19:04:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:04:35.725107 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91436895-2be6-4279-b086-5f661380019d None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:35.793276 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:35.836324 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91436895-2be6-4279-b086-5f661380019d None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:35.871006 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8690b84e-4d1b-463e-84f6-619800a6b362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:35.871668 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8690b84e-4d1b-463e-84f6-619800a6b362 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:35.871950 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8690b84e-4d1b-463e-84f6-619800a6b362 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:35.872225 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8690b84e-4d1b-463e-84f6-619800a6b362 None admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:35.873105 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8690b84e-4d1b-463e-84f6-619800a6b362 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:04:35.873488 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8690b84e-4d1b-463e-84f6-619800a6b362 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:35.874959 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8690b84e-4d1b-463e-84f6-619800a6b362 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:35.890819 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 640/1272] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 2 19:04:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:04:35.896977 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a45d4-e5fb-41dd-9ec7-c411daf98296 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:35.988379 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:35.997612 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a45d4-e5fb-41dd-9ec7-c411daf98296 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:36.030360 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c785467-3263-495b-8cc8-56f0c8707754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:36.031296 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c785467-3263-495b-8cc8-56f0c8707754 None admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:36.031578 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c785467-3263-495b-8cc8-56f0c8707754 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:36.031855 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c785467-3263-495b-8cc8-56f0c8707754 None admin] PATH_INFO: `/v3/projects/8c04020eaebd4647804cf8653f32d9e5/users/73ae837e99764bebbe7a6e06ac9049e3/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:36.033798 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c785467-3263-495b-8cc8-56f0c8707754 None admin] RBAC: Authorizing `identity:create_grant(project_id=8c04020eaebd4647804cf8653f32d9e5, user_id=73ae837e99764bebbe7a6e06ac9049e3, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:36.035266 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c785467-3263-495b-8cc8-56f0c8707754 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:36.154391 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 633/1273] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Jun 2 19:04:35 2022] PUT /identity/v3/projects/8c04020eaebd4647804cf8653f32d9e5/users/73ae837e99764bebbe7a6e06ac9049e3/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:04:38.405373 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06f207c-7b05-4219-9cae-ef991a40f94e None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:38.405882 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06f207c-7b05-4219-9cae-ef991a40f94e None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:38.405971 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06f207c-7b05-4219-9cae-ef991a40f94e None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:38.407386 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 641/1274] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:04:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:04:38.412666 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c9b436-3642-48a4-8a75-9feff9236337 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:38.413067 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c9b436-3642-48a4-8a75-9feff9236337 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:38.413225 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c9b436-3642-48a4-8a75-9feff9236337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:38.429509 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-a8c9b436-3642-48a4-8a75-9feff9236337 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:04:38.430666 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-a8c9b436-3642-48a4-8a75-9feff9236337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:04:38.495699 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 634/1275] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:04:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 02 19:04:38.520824 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06f207c-7b05-4219-9cae-ef991a40f94e None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:38.643443 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06f207c-7b05-4219-9cae-ef991a40f94e None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:38.665725 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:38.686631 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45210c38-6811-4e3f-930b-019a18666792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:38.687650 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45210c38-6811-4e3f-930b-019a18666792 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:38.687943 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45210c38-6811-4e3f-930b-019a18666792 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:38.688223 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45210c38-6811-4e3f-930b-019a18666792 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:38.700547 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45210c38-6811-4e3f-930b-019a18666792 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 02 19:04:38.708497 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-45210c38-6811-4e3f-930b-019a18666792 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 02 19:04:38.708497 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:04:38.708497 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:04:38.708497 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:04:38.708497 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:04:38.708497 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:04:38.708497 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:04:38.708497 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:04:38.708497 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:04:38.708497 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:04:38.708497 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:04:38.708497 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:04:38.708497 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 02 19:04:38.708497 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 02 19:04:38.708497 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 02 19:04:38.708497 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 02 19:04:38.712410 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 02 19:04:38.712410 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 02 19:04:38.712410 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 02 19:04:38.712410 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 02 19:04:38.712410 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:04:38.712410 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:04:38.712410 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:04:38.712410 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:04:38.712410 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:04:38.712410 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:04:38.712410 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:04:38.712410 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:04:38.712410 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:04:38.712410 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:04:38.712410 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:04:38.712410 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:04:38.713321 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:04:38.713321 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:04:38.713321 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:04:38.713321 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:04:38.713321 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 02 19:04:38.713321 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 02 19:04:38.713321 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 02 19:04:38.713321 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 02 19:04:38.713321 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 02 19:04:38.713321 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 02 19:04:38.713321 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 02 19:04:38.713321 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:04:38.713321 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 642/1276] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Jun 2 19:04:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:04:38.716938 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c9b436-3642-48a4-8a75-9feff9236337 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:38.817392 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:38.838100 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c9b436-3642-48a4-8a75-9feff9236337 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:38.875142 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905ff5fd-d6b7-4462-b339-382ef31a0ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:38.875835 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905ff5fd-d6b7-4462-b339-382ef31a0ee9 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:38.876117 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905ff5fd-d6b7-4462-b339-382ef31a0ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:38.876393 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905ff5fd-d6b7-4462-b339-382ef31a0ee9 None admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:38.877159 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905ff5fd-d6b7-4462-b339-382ef31a0ee9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:04:38.877470 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905ff5fd-d6b7-4462-b339-382ef31a0ee9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:38.879158 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905ff5fd-d6b7-4462-b339-382ef31a0ee9 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:38.888431 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 635/1277] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Jun 2 19:04:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:04:38.894486 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45210c38-6811-4e3f-930b-019a18666792 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:39.003906 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45210c38-6811-4e3f-930b-019a18666792 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:39.038319 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613d1f2b-b487-4e66-8a07-b9a18c3300ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:39.039159 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d1f2b-b487-4e66-8a07-b9a18c3300ec None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:39.039504 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d1f2b-b487-4e66-8a07-b9a18c3300ec None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:39.039785 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d1f2b-b487-4e66-8a07-b9a18c3300ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:39.050874 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613d1f2b-b487-4e66-8a07-b9a18c3300ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:39.052340 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613d1f2b-b487-4e66-8a07-b9a18c3300ec None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:39.070581 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-613d1f2b-b487-4e66-8a07-b9a18c3300ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:04:39.070581 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:04:39.070581 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:04:39.070581 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:04:39.070581 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:04:39.070581 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:04:39.070581 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:04:39.070581 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:04:39.070581 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:04:39.070581 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:04:39.070581 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:04:39.070581 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:04:39.070581 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:04:39.070581 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:04:39.070581 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:04:39.071750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:04:39.071750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:04:39.071750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:04:39.071750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:04:39.071750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:04:39.071750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:04:39.071750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:04:39.071750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:04:39.071750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:04:39.071750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:04:39.071750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:04:39.071750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:04:39.071750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:04:39.071750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:04:39.071750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:04:39.071750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:04:39.071750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:04:39.073051 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:04:39.073051 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:04:39.073051 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:04:39.073051 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:04:39.073051 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 643/1278] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:04:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:04:39.077733 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905ff5fd-d6b7-4462-b339-382ef31a0ee9 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:39.182763 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905ff5fd-d6b7-4462-b339-382ef31a0ee9 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:39.215308 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a04805d-8a97-47f5-a622-f25a060c1fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:39.215975 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a04805d-8a97-47f5-a622-f25a060c1fef None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:39.216338 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a04805d-8a97-47f5-a622-f25a060c1fef None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:39.216617 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a04805d-8a97-47f5-a622-f25a060c1fef None admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:39.217334 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a04805d-8a97-47f5-a622-f25a060c1fef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:04:39.217638 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a04805d-8a97-47f5-a622-f25a060c1fef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:39.219022 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a04805d-8a97-47f5-a622-f25a060c1fef None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:39.230902 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 636/1279] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:04:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:04:39.236527 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613d1f2b-b487-4e66-8a07-b9a18c3300ec None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:39.268281 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:39.337927 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613d1f2b-b487-4e66-8a07-b9a18c3300ec None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:39.371336 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137004f1-f8a2-4866-99f5-cbea40ca8be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:39.372133 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137004f1-f8a2-4866-99f5-cbea40ca8be5 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:39.372421 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137004f1-f8a2-4866-99f5-cbea40ca8be5 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:39.372692 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137004f1-f8a2-4866-99f5-cbea40ca8be5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:39.381904 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137004f1-f8a2-4866-99f5-cbea40ca8be5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:39.383397 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137004f1-f8a2-4866-99f5-cbea40ca8be5 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:39.397971 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-137004f1-f8a2-4866-99f5-cbea40ca8be5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 02 19:04:39.397971 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:04:39.397971 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:04:39.397971 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:04:39.397971 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:04:39.397971 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:04:39.397971 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:04:39.397971 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:04:39.397971 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:04:39.397971 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:04:39.397971 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:04:39.397971 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:04:39.397971 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 02 19:04:39.397971 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 02 19:04:39.397971 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 02 19:04:39.397971 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 02 19:04:39.399848 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:04:39.399848 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:04:39.399848 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 02 19:04:39.399848 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:04:39.399848 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 02 19:04:39.399848 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 02 19:04:39.399848 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:04:39.399848 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:04:39.399848 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:04:39.399848 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:04:39.399848 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:04:39.399848 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:04:39.399848 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:04:39.399848 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:04:39.399848 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:04:39.399848 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:04:39.399848 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:04:39.399848 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:04:39.400883 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:04:39.400883 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:04:39.400883 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 02 19:04:39.400883 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 02 19:04:39.400883 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 02 19:04:39.400883 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 02 19:04:39.400883 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 02 19:04:39.400883 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 02 19:04:39.400883 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 02 19:04:39.400883 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:04:39.400883 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 644/1280] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:04:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:04:39.405047 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a04805d-8a97-47f5-a622-f25a060c1fef None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:39.408936 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:39.499937 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a04805d-8a97-47f5-a622-f25a060c1fef None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:39.537753 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf11586-edaa-4cc2-99f9-51bb809a4043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:39.538695 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf11586-edaa-4cc2-99f9-51bb809a4043 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:39.539076 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf11586-edaa-4cc2-99f9-51bb809a4043 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:39.539360 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf11586-edaa-4cc2-99f9-51bb809a4043 None admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:39.540190 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf11586-edaa-4cc2-99f9-51bb809a4043 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:04:39.540496 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf11586-edaa-4cc2-99f9-51bb809a4043 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:39.542024 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf11586-edaa-4cc2-99f9-51bb809a4043 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:39.560270 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 637/1281] 10.222.0.33 () {56 vars in 1143 bytes} [Thu Jun 2 19:04:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:04:39.566142 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137004f1-f8a2-4866-99f5-cbea40ca8be5 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:39.655802 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:39.665856 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137004f1-f8a2-4866-99f5-cbea40ca8be5 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:39.700194 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942bbaef-ca86-4c7a-9575-7ebec200916e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:39.700971 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942bbaef-ca86-4c7a-9575-7ebec200916e None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:39.701354 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942bbaef-ca86-4c7a-9575-7ebec200916e None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:39.701717 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942bbaef-ca86-4c7a-9575-7ebec200916e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:39.769544 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942bbaef-ca86-4c7a-9575-7ebec200916e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:39.771039 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942bbaef-ca86-4c7a-9575-7ebec200916e None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:39.862882 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:39.872745 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 645/1282] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Jun 2 19:04:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 02 19:04:39.878823 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf11586-edaa-4cc2-99f9-51bb809a4043 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:39.938483 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:39.975474 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf11586-edaa-4cc2-99f9-51bb809a4043 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:40.008886 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e22a70-4d4e-4b53-a468-bd438833a81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:40.009729 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e22a70-4d4e-4b53-a468-bd438833a81c None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:40.010015 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e22a70-4d4e-4b53-a468-bd438833a81c None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:40.010281 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e22a70-4d4e-4b53-a468-bd438833a81c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:40.020007 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e22a70-4d4e-4b53-a468-bd438833a81c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:40.021539 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e22a70-4d4e-4b53-a468-bd438833a81c None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:40.039526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-63e22a70-4d4e-4b53-a468-bd438833a81c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:04:40.039526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:04:40.039526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:04:40.039526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:04:40.039526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:04:40.039526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:04:40.039526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:04:40.039526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:04:40.039526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:04:40.039526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:04:40.039526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:04:40.039526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:04:40.039526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 02 19:04:40.039526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 02 19:04:40.039526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 02 19:04:40.040576 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 02 19:04:40.040576 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:04:40.040576 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:04:40.040576 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:04:40.040576 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:04:40.040576 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:04:40.040576 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:04:40.040576 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:04:40.040576 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:04:40.040576 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:04:40.040576 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:04:40.040576 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:04:40.040576 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:04:40.040576 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:04:40.040576 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:04:40.040576 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:04:40.040576 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:04:40.041515 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 02 19:04:40.041515 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 02 19:04:40.041515 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 02 19:04:40.041515 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:04:40.041515 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 638/1283] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:04:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:04:40.046436 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942bbaef-ca86-4c7a-9575-7ebec200916e None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:40.145464 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942bbaef-ca86-4c7a-9575-7ebec200916e None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:40.179279 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04aa0d1-4312-4598-895a-ba87d5f54a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:40.179936 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04aa0d1-4312-4598-895a-ba87d5f54a7f None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:40.180218 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04aa0d1-4312-4598-895a-ba87d5f54a7f None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:40.180490 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04aa0d1-4312-4598-895a-ba87d5f54a7f None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:40.181206 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04aa0d1-4312-4598-895a-ba87d5f54a7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:04:40.181567 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04aa0d1-4312-4598-895a-ba87d5f54a7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:40.182955 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04aa0d1-4312-4598-895a-ba87d5f54a7f None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:40.193993 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 646/1284] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Jun 2 19:04:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:04:40.199890 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e22a70-4d4e-4b53-a468-bd438833a81c None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:40.298743 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e22a70-4d4e-4b53-a468-bd438833a81c None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:40.312772 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:40.332248 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe8b1ad-32d2-4574-a1d4-369f90ce0fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:40.333040 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe8b1ad-32d2-4574-a1d4-369f90ce0fc3 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:40.333322 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe8b1ad-32d2-4574-a1d4-369f90ce0fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:40.333599 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe8b1ad-32d2-4574-a1d4-369f90ce0fc3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:40.340174 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe8b1ad-32d2-4574-a1d4-369f90ce0fc3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:40.341618 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe8b1ad-32d2-4574-a1d4-369f90ce0fc3 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:40.355613 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application [None req-bbe8b1ad-32d2-4574-a1d4-369f90ce0fc3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:04:40.355613 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:04:40.355613 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:04:40.355613 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:04:40.355613 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:04:40.355613 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:04:40.355613 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:04:40.355613 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:04:40.355613 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:04:40.355613 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:04:40.355613 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:04:40.355613 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:04:40.355613 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 02 19:04:40.355613 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 02 19:04:40.355613 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 02 19:04:40.356682 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 02 19:04:40.356682 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:04:40.356682 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:04:40.356682 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:04:40.356682 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:04:40.356682 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:04:40.356682 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:04:40.356682 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:04:40.356682 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application with Lock( Jun 02 19:04:40.356682 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:04:40.356682 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:04:40.356682 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:04:40.356682 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:04:40.356682 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:04:40.356682 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:04:40.356682 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:04:40.356682 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:04:40.358004 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 02 19:04:40.358004 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 02 19:04:40.358004 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 02 19:04:40.358004 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 02 19:04:40.358004 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 02 19:04:40.358004 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 02 19:04:40.358004 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 02 19:04:40.358004 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: ERROR keystone.server.flask.application Jun 02 19:04:40.358004 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 639/1285] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Jun 2 19:04:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:04:40.362956 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04aa0d1-4312-4598-895a-ba87d5f54a7f None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:40.418398 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:40.468818 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04aa0d1-4312-4598-895a-ba87d5f54a7f None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:40.505557 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa115f7a-0779-4954-94a3-25213fdba1b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:40.506773 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa115f7a-0779-4954-94a3-25213fdba1b6 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:40.507063 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa115f7a-0779-4954-94a3-25213fdba1b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:40.507551 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa115f7a-0779-4954-94a3-25213fdba1b6 None admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:40.508403 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa115f7a-0779-4954-94a3-25213fdba1b6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:04:40.508721 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa115f7a-0779-4954-94a3-25213fdba1b6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:40.510206 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa115f7a-0779-4954-94a3-25213fdba1b6 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:40.524734 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 647/1286] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Jun 2 19:04:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:04:40.530876 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe8b1ad-32d2-4574-a1d4-369f90ce0fc3 None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:40.629036 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe8b1ad-32d2-4574-a1d4-369f90ce0fc3 None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:40.662879 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3665b5-73d5-4c10-8448-aac9eb4229db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:40.663710 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3665b5-73d5-4c10-8448-aac9eb4229db None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:40.663988 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3665b5-73d5-4c10-8448-aac9eb4229db None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:40.664261 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3665b5-73d5-4c10-8448-aac9eb4229db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:40.665084 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3665b5-73d5-4c10-8448-aac9eb4229db None admin] RBAC: Adding query filter params (role.id=f13fdd0847924014abf1e95594195e78, scope.project.id=8c04020eaebd4647804cf8653f32d9e5, user.id=73ae837e99764bebbe7a6e06ac9049e3) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:04:40.665397 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3665b5-73d5-4c10-8448-aac9eb4229db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:40.666843 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3665b5-73d5-4c10-8448-aac9eb4229db None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:40.672688 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 640/1287] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Jun 2 19:04:40 2022] GET /identity/v3/role_assignments?user.id=73ae837e99764bebbe7a6e06ac9049e3&scope.project.id=8c04020eaebd4647804cf8653f32d9e5&role.id=f13fdd0847924014abf1e95594195e78 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:04:42.979947 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f6ad81-ebae-4d51-96ca-a99713018e48 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:42.980485 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f6ad81-ebae-4d51-96ca-a99713018e48 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:42.980485 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f6ad81-ebae-4d51-96ca-a99713018e48 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:42.981910 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 648/1288] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:04:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:04:42.987421 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8a11de-07c4-40ce-8ad8-c6ed71fca34b None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:42.987824 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8a11de-07c4-40ce-8ad8-c6ed71fca34b None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:42.987992 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8a11de-07c4-40ce-8ad8-c6ed71fca34b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:43.004936 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-ac8a11de-07c4-40ce-8ad8-c6ed71fca34b None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:04:43.006623 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-ac8a11de-07c4-40ce-8ad8-c6ed71fca34b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:04:43.036219 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:43.072140 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 641/1289] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:04:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 02 19:04:43.095119 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f6ad81-ebae-4d51-96ca-a99713018e48 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:43.200002 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f6ad81-ebae-4d51-96ca-a99713018e48 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:43.211947 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:43.235116 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552b450b-2f21-4085-b3ac-9ce90350e8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:43.235856 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b450b-2f21-4085-b3ac-9ce90350e8ff None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:43.236142 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b450b-2f21-4085-b3ac-9ce90350e8ff None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:43.236420 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b450b-2f21-4085-b3ac-9ce90350e8ff None admin] PATH_INFO: `/v3/services/placement` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:43.237073 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552b450b-2f21-4085-b3ac-9ce90350e8ff None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:43.238524 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552b450b-2f21-4085-b3ac-9ce90350e8ff None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:43.250497 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-552b450b-2f21-4085-b3ac-9ce90350e8ff None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 02 19:04:43.250497 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:04:43.250497 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 02 19:04:43.250497 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 02 19:04:43.250497 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 02 19:04:43.250497 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 02 19:04:43.250497 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 02 19:04:43.250497 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 02 19:04:43.250497 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 02 19:04:43.250497 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:04:43.250497 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 02 19:04:43.250497 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:04:43.250497 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:04:43.250497 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:04:43.250497 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:04:43.250497 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:04:43.250497 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:04:43.251818 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:04:43.251818 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:04:43.251818 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:04:43.251818 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:04:43.251818 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:04:43.251818 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:04:43.251818 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 02 19:04:43.251818 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 02 19:04:43.251818 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 02 19:04:43.251818 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 02 19:04:43.251818 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:04:43.251818 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:04:43.251818 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:04:43.251818 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:04:43.251818 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:04:43.251818 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:04:43.253004 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:04:43.253004 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:04:43.253004 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:04:43.253004 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:04:43.253004 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:04:43.253004 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:04:43.253004 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:04:43.253004 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:04:43.253004 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:04:43.253004 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:04:43.253004 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 02 19:04:43.253004 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 02 19:04:43.253004 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 02 19:04:43.253004 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:04:43.253004 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 649/1290] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Jun 2 19:04:43 2022] GET /identity/v3/services/placement => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:04:43.258341 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8a11de-07c4-40ce-8ad8-c6ed71fca34b None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:43.354486 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8a11de-07c4-40ce-8ad8-c6ed71fca34b None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:43.389102 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46559657-1101-4845-a498-7a7ccdf83033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:43.389760 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46559657-1101-4845-a498-7a7ccdf83033 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:43.390043 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46559657-1101-4845-a498-7a7ccdf83033 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:43.390318 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46559657-1101-4845-a498-7a7ccdf83033 None admin] PATH_INFO: `/v3/services` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:43.391054 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46559657-1101-4845-a498-7a7ccdf83033 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:04:43.396079 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46559657-1101-4845-a498-7a7ccdf83033 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:43.396079 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46559657-1101-4845-a498-7a7ccdf83033 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:43.397967 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 642/1291] 10.222.0.33 () {56 vars in 1116 bytes} [Thu Jun 2 19:04:43 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:04:43.403843 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552b450b-2f21-4085-b3ac-9ce90350e8ff None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:43.502070 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552b450b-2f21-4085-b3ac-9ce90350e8ff None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:43.534195 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c8dc9-7729-4c86-9d82-7e1bb1c75821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:43.534892 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c8dc9-7729-4c86-9d82-7e1bb1c75821 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:43.535176 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c8dc9-7729-4c86-9d82-7e1bb1c75821 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:43.535453 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c8dc9-7729-4c86-9d82-7e1bb1c75821 None admin] PATH_INFO: `/v3/services` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:43.536157 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2c8dc9-7729-4c86-9d82-7e1bb1c75821 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:04:43.536463 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2c8dc9-7729-4c86-9d82-7e1bb1c75821 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:43.538405 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2c8dc9-7729-4c86-9d82-7e1bb1c75821 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:43.543291 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 650/1292] 10.222.0.33 () {56 vars in 1116 bytes} [Thu Jun 2 19:04:43 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:04:45.812591 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d96e55-d3e3-4238-85bd-31a34f98347c None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:45.813154 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d96e55-d3e3-4238-85bd-31a34f98347c None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:45.813154 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d96e55-d3e3-4238-85bd-31a34f98347c None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:45.814505 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 643/1293] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:04:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:04:45.819821 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc22418-9616-4869-a396-2807bb2750ff None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:45.820196 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc22418-9616-4869-a396-2807bb2750ff None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:45.820374 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc22418-9616-4869-a396-2807bb2750ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:45.837189 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-1cc22418-9616-4869-a396-2807bb2750ff None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:04:45.838415 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-1cc22418-9616-4869-a396-2807bb2750ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:04:45.868736 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:45.905126 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 651/1294] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:04:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 02 19:04:45.929372 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d96e55-d3e3-4238-85bd-31a34f98347c None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:45.960277 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:46.027769 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d96e55-d3e3-4238-85bd-31a34f98347c None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:46.061316 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e45641-056c-492b-8db1-4f468638743c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:46.062004 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e45641-056c-492b-8db1-4f468638743c None admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:46.062286 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e45641-056c-492b-8db1-4f468638743c None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:46.062562 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e45641-056c-492b-8db1-4f468638743c None admin] PATH_INFO: `/v3/services` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:46.066294 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e45641-056c-492b-8db1-4f468638743c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:46.066294 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e45641-056c-492b-8db1-4f468638743c None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:46.165575 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 644/1295] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Jun 2 19:04:45 2022] POST /identity/v3/services => generated 246 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:04:48.467804 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03823d3-3830-478f-bd5a-c0e2ae587417 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:48.468323 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03823d3-3830-478f-bd5a-c0e2ae587417 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:48.468323 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03823d3-3830-478f-bd5a-c0e2ae587417 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:48.469637 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 652/1296] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:04:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:04:48.475890 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5239cd-094e-4b05-9760-0940ec551196 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:48.476311 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5239cd-094e-4b05-9760-0940ec551196 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:48.476452 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5239cd-094e-4b05-9760-0940ec551196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:48.493796 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-bc5239cd-094e-4b05-9760-0940ec551196 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:04:48.495077 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-bc5239cd-094e-4b05-9760-0940ec551196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:04:48.570385 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 645/1297] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:04:48 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 02 19:04:48.594023 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03823d3-3830-478f-bd5a-c0e2ae587417 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:48.705643 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03823d3-3830-478f-bd5a-c0e2ae587417 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:48.743462 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df89d1f-c1ea-431d-a2aa-fbf29d80603f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:48.744279 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df89d1f-c1ea-431d-a2aa-fbf29d80603f None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:48.744562 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df89d1f-c1ea-431d-a2aa-fbf29d80603f None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:48.744839 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df89d1f-c1ea-431d-a2aa-fbf29d80603f None admin] PATH_INFO: `/v3/services/placement` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:48.745502 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df89d1f-c1ea-431d-a2aa-fbf29d80603f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:48.746921 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df89d1f-c1ea-431d-a2aa-fbf29d80603f None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:48.759744 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-5df89d1f-c1ea-431d-a2aa-fbf29d80603f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 02 19:04:48.759744 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:04:48.759744 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 02 19:04:48.759744 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 02 19:04:48.759744 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 02 19:04:48.759744 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 02 19:04:48.759744 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 02 19:04:48.759744 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 02 19:04:48.759744 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 02 19:04:48.759744 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:04:48.759744 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 02 19:04:48.759744 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:04:48.759744 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:04:48.759744 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:04:48.759744 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:04:48.759744 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:04:48.759744 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:04:48.761021 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:04:48.761021 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:04:48.761021 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:04:48.761021 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:04:48.761021 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:04:48.761021 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:04:48.761021 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 02 19:04:48.761021 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 02 19:04:48.761021 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 02 19:04:48.761021 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 02 19:04:48.761021 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:04:48.761021 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:04:48.761021 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:04:48.761021 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:04:48.761021 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:04:48.761021 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:04:48.762096 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:04:48.762096 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:04:48.762096 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:04:48.762096 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:04:48.762096 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:04:48.762096 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:04:48.762096 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:04:48.762096 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:04:48.762096 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:04:48.762096 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:04:48.762096 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 02 19:04:48.762096 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 02 19:04:48.762096 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 02 19:04:48.762096 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:04:48.762096 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 653/1298] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Jun 2 19:04:48 2022] GET /identity/v3/services/placement => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:04:48.767592 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5239cd-094e-4b05-9760-0940ec551196 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:48.787076 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:48.868935 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5239cd-094e-4b05-9760-0940ec551196 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:48.905104 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ab2a4e-1581-447b-b543-cbdd450d0c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:48.905772 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ab2a4e-1581-447b-b543-cbdd450d0c5d None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:48.906052 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ab2a4e-1581-447b-b543-cbdd450d0c5d None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:48.906326 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ab2a4e-1581-447b-b543-cbdd450d0c5d None admin] PATH_INFO: `/v3/services` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:48.907066 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ab2a4e-1581-447b-b543-cbdd450d0c5d None admin] RBAC: Adding query filter params (name=placement) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:04:48.907371 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ab2a4e-1581-447b-b543-cbdd450d0c5d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:48.908744 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ab2a4e-1581-447b-b543-cbdd450d0c5d None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:48.915198 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 646/1299] 10.222.0.33 () {56 vars in 1116 bytes} [Thu Jun 2 19:04:48 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:04:48.921681 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df89d1f-c1ea-431d-a2aa-fbf29d80603f None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:48.954881 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:49.023367 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df89d1f-c1ea-431d-a2aa-fbf29d80603f None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:49.058815 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2287719-aa7e-474f-b485-1968db9be7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:49.059489 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2287719-aa7e-474f-b485-1968db9be7cd None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:49.059776 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2287719-aa7e-474f-b485-1968db9be7cd None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:49.060049 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2287719-aa7e-474f-b485-1968db9be7cd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:49.060846 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2287719-aa7e-474f-b485-1968db9be7cd None admin] RBAC: Adding query filter params (interface=public, service_id=374cdbd574cd43ea8dfdf9e57c9444d4, region_id=RegionOne) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:04:49.061160 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2287719-aa7e-474f-b485-1968db9be7cd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:49.062603 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2287719-aa7e-474f-b485-1968db9be7cd None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:49.068034 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 654/1300] 10.222.0.33 () {56 vars in 1251 bytes} [Thu Jun 2 19:04:48 2022] GET /identity/v3/endpoints?service_id=374cdbd574cd43ea8dfdf9e57c9444d4&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:04:49.073855 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ab2a4e-1581-447b-b543-cbdd450d0c5d None admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:49.163579 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:49.181389 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ab2a4e-1581-447b-b543-cbdd450d0c5d None admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:49.216302 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83225dc-0c50-486f-bdcd-5ac8dd00d44f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:49.216965 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83225dc-0c50-486f-bdcd-5ac8dd00d44f None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:49.217250 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83225dc-0c50-486f-bdcd-5ac8dd00d44f None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:49.217528 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83225dc-0c50-486f-bdcd-5ac8dd00d44f None admin] PATH_INFO: `/v3/services` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:49.218216 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83225dc-0c50-486f-bdcd-5ac8dd00d44f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:49.219612 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83225dc-0c50-486f-bdcd-5ac8dd00d44f None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:49.227027 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 647/1301] 10.222.0.33 () {56 vars in 1087 bytes} [Thu Jun 2 19:04:49 2022] GET /identity/v3/services => generated 2191 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:04:51.501112 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe6001-d87d-4530-88ea-5bd492d87352 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:51.501930 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe6001-d87d-4530-88ea-5bd492d87352 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:51.501930 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe6001-d87d-4530-88ea-5bd492d87352 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:51.503339 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 655/1302] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:04:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:04:51.509427 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ef742-3422-43dc-a77c-eabc5a842b07 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:51.509865 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ef742-3422-43dc-a77c-eabc5a842b07 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:51.509982 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ef742-3422-43dc-a77c-eabc5a842b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:51.527329 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-353ef742-3422-43dc-a77c-eabc5a842b07 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:04:51.528511 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-353ef742-3422-43dc-a77c-eabc5a842b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:04:51.594376 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 648/1303] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:04:51 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 02 19:04:51.617871 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbe6001-d87d-4530-88ea-5bd492d87352 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:51.717621 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:51.725683 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbe6001-d87d-4530-88ea-5bd492d87352 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:51.763717 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd03068-e912-449e-be54-af6c07dfeb2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:51.764449 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd03068-e912-449e-be54-af6c07dfeb2c None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:51.764732 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd03068-e912-449e-be54-af6c07dfeb2c None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:51.765002 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd03068-e912-449e-be54-af6c07dfeb2c None admin] PATH_INFO: `/v3/services/placement` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:51.765661 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd03068-e912-449e-be54-af6c07dfeb2c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:51.767062 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd03068-e912-449e-be54-af6c07dfeb2c None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:51.779574 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-cbd03068-e912-449e-be54-af6c07dfeb2c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 02 19:04:51.779574 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:04:51.779574 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 02 19:04:51.779574 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 02 19:04:51.779574 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 02 19:04:51.779574 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 02 19:04:51.779574 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 02 19:04:51.779574 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 02 19:04:51.779574 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 02 19:04:51.779574 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:04:51.779574 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 02 19:04:51.779574 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:04:51.779574 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 02 19:04:51.779574 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 02 19:04:51.779574 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 02 19:04:51.779574 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 02 19:04:51.779574 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 02 19:04:51.780740 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 02 19:04:51.780740 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 02 19:04:51.780740 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 02 19:04:51.780740 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 02 19:04:51.780740 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 02 19:04:51.780740 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 02 19:04:51.780740 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 02 19:04:51.780740 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 02 19:04:51.780740 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 02 19:04:51.780740 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 02 19:04:51.780740 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 02 19:04:51.780740 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 02 19:04:51.780740 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 02 19:04:51.780740 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 02 19:04:51.780740 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 02 19:04:51.780740 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Jun 02 19:04:51.781591 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 02 19:04:51.781591 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Jun 02 19:04:51.781591 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 02 19:04:51.781591 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Jun 02 19:04:51.781591 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 02 19:04:51.781591 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 02 19:04:51.781591 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 02 19:04:51.781591 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Jun 02 19:04:51.781591 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 02 19:04:51.781591 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Jun 02 19:04:51.781591 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 02 19:04:51.781591 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 02 19:04:51.781591 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 02 19:04:51.781591 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Jun 02 19:04:51.781591 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 656/1304] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Jun 2 19:04:51 2022] GET /identity/v3/services/placement => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 02 19:04:51.787486 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353ef742-3422-43dc-a77c-eabc5a842b07 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:51.866968 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:04:51.889496 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353ef742-3422-43dc-a77c-eabc5a842b07 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:51.923708 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5abe6f5-4739-402a-ba4d-2ca9f86f3fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:51.924367 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5abe6f5-4739-402a-ba4d-2ca9f86f3fa8 None admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:51.924675 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5abe6f5-4739-402a-ba4d-2ca9f86f3fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:51.924950 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5abe6f5-4739-402a-ba4d-2ca9f86f3fa8 None admin] PATH_INFO: `/v3/services` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:51.925669 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5abe6f5-4739-402a-ba4d-2ca9f86f3fa8 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:04:51.925965 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5abe6f5-4739-402a-ba4d-2ca9f86f3fa8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:51.927958 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5abe6f5-4739-402a-ba4d-2ca9f86f3fa8 None admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:51.933120 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 649/1305] 10.222.0.33 () {56 vars in 1116 bytes} [Thu Jun 2 19:04:51 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:04:51.939203 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd03068-e912-449e-be54-af6c07dfeb2c None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:04:52.040375 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd03068-e912-449e-be54-af6c07dfeb2c None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:04:52.074105 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df8da8c-a011-4e82-a481-db025f967f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:04:52.074790 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df8da8c-a011-4e82-a481-db025f967f9f None admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:04:52.075083 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df8da8c-a011-4e82-a481-db025f967f9f None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:04:52.075390 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df8da8c-a011-4e82-a481-db025f967f9f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:04:52.076136 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df8da8c-a011-4e82-a481-db025f967f9f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:04:52.077547 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df8da8c-a011-4e82-a481-db025f967f9f None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:04:52.172118 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 657/1306] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Jun 2 19:04:51 2022] POST /identity/v3/endpoints => generated 331 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:05:01.930053 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=30 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:05:02.166701 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:12:01.794646 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f928bea8-8ce7-4d03-a7b4-92d7d36ec09b None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:12:01.795238 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f928bea8-8ce7-4d03-a7b4-92d7d36ec09b None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:12:01.795238 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f928bea8-8ce7-4d03-a7b4-92d7d36ec09b None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:12:01.802114 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee9c495-951c-4717-ad8f-d165c804493a None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:12:01.802594 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee9c495-951c-4717-ad8f-d165c804493a None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:12:01.802682 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee9c495-951c-4717-ad8f-d165c804493a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:12:01.806133 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 650/1307] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:12:01 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:12:01.886908 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-1ee9c495-951c-4717-ad8f-d165c804493a None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:12:01.888679 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-1ee9c495-951c-4717-ad8f-d165c804493a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:12:02.004130 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 658/1308] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:12:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 19:12:02.072639 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f8c37d-269e-43fe-8040-a4e226b29fef None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:12:02.072878 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f8c37d-269e-43fe-8040-a4e226b29fef None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:12:02.073161 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f8c37d-269e-43fe-8040-a4e226b29fef None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:12:02.074617 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 651/1309] 10.222.0.33 () {54 vars in 964 bytes} [Thu Jun 2 19:12:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:12:02.085356 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb5735-6e81-4983-a4f4-b4504d358a40 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:12:02.085660 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb5735-6e81-4983-a4f4-b4504d358a40 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:12:02.085938 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb5735-6e81-4983-a4f4-b4504d358a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:12:02.249343 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-35fb5735-6e81-4983-a4f4-b4504d358a40 None None] MFA Rules not processed for user `7417a54046e34e899d7b03f2f2314deb`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:12:02.250789 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-35fb5735-6e81-4983-a4f4-b4504d358a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:12:02.294115 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:12:02.379013 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 659/1310] 10.222.0.33 () {58 vars in 1061 bytes} [Thu Jun 2 19:12:02 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:12:02.379858 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f8c37d-269e-43fe-8040-a4e226b29fef None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:12:02.381701 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-86f8c37d-269e-43fe-8040-a4e226b29fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:12:02.510289 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f8c37d-269e-43fe-8040-a4e226b29fef None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:12:02.591258 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46712742-a175-4784-ad38-a54300a96def service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:12:02.591953 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46712742-a175-4784-ad38-a54300a96def service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:12:02.592241 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46712742-a175-4784-ad38-a54300a96def service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:12:02.592522 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46712742-a175-4784-ad38-a54300a96def service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:12:02.594186 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-46712742-a175-4784-ad38-a54300a96def service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:12:02.651749 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:12:02.712017 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46712742-a175-4784-ad38-a54300a96def service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:12:02.713652 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46712742-a175-4784-ad38-a54300a96def service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:12:02.734312 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 652/1311] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:12:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:12:05.398060 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e6fc3c-9047-4e7e-a1f6-28dbde262148 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:12:05.398572 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e6fc3c-9047-4e7e-a1f6-28dbde262148 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:12:05.398572 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e6fc3c-9047-4e7e-a1f6-28dbde262148 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:12:05.399931 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 660/1312] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:12:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:12:05.405196 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2de28d-dd47-4073-b65a-98120072fc3a None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:12:05.405644 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2de28d-dd47-4073-b65a-98120072fc3a None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:12:05.405764 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2de28d-dd47-4073-b65a-98120072fc3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:12:05.487915 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-7b2de28d-dd47-4073-b65a-98120072fc3a None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:12:05.489136 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-7b2de28d-dd47-4073-b65a-98120072fc3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:12:05.594321 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 653/1313] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:12:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:12:05.633530 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e6fc3c-9047-4e7e-a1f6-28dbde262148 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:12:05.635213 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-42e6fc3c-9047-4e7e-a1f6-28dbde262148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:12:05.756699 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e6fc3c-9047-4e7e-a1f6-28dbde262148 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:12:05.972920 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:12:05.975567 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf6406b-fb21-4f1f-a420-de6411f530d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:12:05.976229 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf6406b-fb21-4f1f-a420-de6411f530d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:12:05.976514 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf6406b-fb21-4f1f-a420-de6411f530d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:12:05.976789 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf6406b-fb21-4f1f-a420-de6411f530d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:12:05.978460 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-cbf6406b-fb21-4f1f-a420-de6411f530d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:12:06.092720 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf6406b-fb21-4f1f-a420-de6411f530d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:12:06.094093 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf6406b-fb21-4f1f-a420-de6411f530d4 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:12:06.114473 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 661/1314] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:12:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:12:09.000966 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8635023-96e6-4e74-a9b6-581fc16b9247 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:12:09.001544 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8635023-96e6-4e74-a9b6-581fc16b9247 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:12:09.001544 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8635023-96e6-4e74-a9b6-581fc16b9247 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:12:09.002928 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 654/1315] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:12:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:12:09.008094 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69be851-d852-4660-9a60-daa339cab98c None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:12:09.008526 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69be851-d852-4660-9a60-daa339cab98c None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:12:09.008638 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69be851-d852-4660-9a60-daa339cab98c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:12:09.072106 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-f69be851-d852-4660-9a60-daa339cab98c None None] MFA Rules not processed for user `720bc045e0bf473dbe5912892ff7b8e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:12:09.073339 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-f69be851-d852-4660-9a60-daa339cab98c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:12:09.124894 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:12:09.170437 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 662/1316] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:12:09 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:12:09.210090 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8635023-96e6-4e74-a9b6-581fc16b9247 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:12:09.211706 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-b8635023-96e6-4e74-a9b6-581fc16b9247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:12:09.222082 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:12:09.315357 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8635023-96e6-4e74-a9b6-581fc16b9247 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:12:09.389725 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8834811a-8763-44dc-8857-6573b3e4617f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:12:09.390368 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8834811a-8763-44dc-8857-6573b3e4617f service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:12:09.390653 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8834811a-8763-44dc-8857-6573b3e4617f service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:12:09.390966 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8834811a-8763-44dc-8857-6573b3e4617f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:12:09.392620 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-8834811a-8763-44dc-8857-6573b3e4617f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:12:09.454843 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:12:09.503321 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8834811a-8763-44dc-8857-6573b3e4617f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:12:09.504830 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8834811a-8763-44dc-8857-6573b3e4617f service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:12:09.523580 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 655/1317] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:12:09 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:12:12.744918 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d234a1-df5a-4aed-86d9-77ba7fd43de1 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:12:12.745500 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d234a1-df5a-4aed-86d9-77ba7fd43de1 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:12:12.745500 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d234a1-df5a-4aed-86d9-77ba7fd43de1 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:12:12.746932 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 663/1318] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:12:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:12:12.752904 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da0cdc-6986-4eb7-b650-5472e507e157 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:12:12.753320 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da0cdc-6986-4eb7-b650-5472e507e157 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:12:12.753442 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da0cdc-6986-4eb7-b650-5472e507e157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:12:12.818544 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-d3da0cdc-6986-4eb7-b650-5472e507e157 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:12:12.819777 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-d3da0cdc-6986-4eb7-b650-5472e507e157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:12:12.914782 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 656/1319] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:12:12 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 19:12:12.954268 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d234a1-df5a-4aed-86d9-77ba7fd43de1 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:12:12.955828 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-14d234a1-df5a-4aed-86d9-77ba7fd43de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:12:13.057642 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d234a1-df5a-4aed-86d9-77ba7fd43de1 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:12:13.133843 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8a712a-5585-493c-910f-9bbc7b09d7b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:12:13.134496 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8a712a-5585-493c-910f-9bbc7b09d7b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:12:13.134806 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8a712a-5585-493c-910f-9bbc7b09d7b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:12:13.135087 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8a712a-5585-493c-910f-9bbc7b09d7b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:12:13.136808 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-aa8a712a-5585-493c-910f-9bbc7b09d7b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:12:13.144656 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:12:13.245162 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8a712a-5585-493c-910f-9bbc7b09d7b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:12:13.246610 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8a712a-5585-493c-910f-9bbc7b09d7b3 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:12:13.265745 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 664/1320] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:12:12 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:12:16.624030 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870f5315-031f-45ee-9093-b31a662f11a8 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:12:16.624604 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870f5315-031f-45ee-9093-b31a662f11a8 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:12:16.624604 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870f5315-031f-45ee-9093-b31a662f11a8 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:12:16.625957 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 657/1321] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:12:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:12:16.631441 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd3f4c0-0b06-43fb-96f5-0118e0682d5c None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:12:16.631898 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd3f4c0-0b06-43fb-96f5-0118e0682d5c None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:12:16.636819 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd3f4c0-0b06-43fb-96f5-0118e0682d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:12:16.698642 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-fbd3f4c0-0b06-43fb-96f5-0118e0682d5c None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:12:16.699892 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-fbd3f4c0-0b06-43fb-96f5-0118e0682d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:12:16.762856 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:12:16.803370 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 665/1322] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:12:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:12:16.840215 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870f5315-031f-45ee-9093-b31a662f11a8 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:12:16.842394 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-870f5315-031f-45ee-9093-b31a662f11a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:12:16.861676 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:12:16.946989 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870f5315-031f-45ee-9093-b31a662f11a8 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:12:17.022899 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff72d79b-cd0c-43e2-9b59-a62fcad4b017 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:12:17.023553 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff72d79b-cd0c-43e2-9b59-a62fcad4b017 service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:12:17.023859 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff72d79b-cd0c-43e2-9b59-a62fcad4b017 service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:12:17.024113 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff72d79b-cd0c-43e2-9b59-a62fcad4b017 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:12:17.025787 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-ff72d79b-cd0c-43e2-9b59-a62fcad4b017 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:12:17.096497 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:12:17.136930 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff72d79b-cd0c-43e2-9b59-a62fcad4b017 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:12:17.139060 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff72d79b-cd0c-43e2-9b59-a62fcad4b017 service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:12:17.157452 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 658/1323] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:12:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:12:19.503898 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68161e78-a474-4460-80dd-669e3fb6dd11 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:12:19.504478 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68161e78-a474-4460-80dd-669e3fb6dd11 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:12:19.504478 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68161e78-a474-4460-80dd-669e3fb6dd11 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:12:19.505891 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 666/1324] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:12:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:12:19.511311 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c49f25-1507-4cf9-bae2-3ca81d8859d4 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:12:19.511728 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c49f25-1507-4cf9-bae2-3ca81d8859d4 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:12:19.512869 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c49f25-1507-4cf9-bae2-3ca81d8859d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:12:19.581950 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-c1c49f25-1507-4cf9-bae2-3ca81d8859d4 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:12:19.583186 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-c1c49f25-1507-4cf9-bae2-3ca81d8859d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:12:19.692161 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 659/1325] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:12:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:12:19.728705 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68161e78-a474-4460-80dd-669e3fb6dd11 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:12:19.730895 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-68161e78-a474-4460-80dd-669e3fb6dd11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:12:19.836166 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68161e78-a474-4460-80dd-669e3fb6dd11 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:12:19.913370 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ff2821-2948-469b-aef8-aa51cec28474 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:12:19.913995 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff2821-2948-469b-aef8-aa51cec28474 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:12:19.914274 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff2821-2948-469b-aef8-aa51cec28474 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:12:19.914546 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff2821-2948-469b-aef8-aa51cec28474 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:12:19.916228 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-35ff2821-2948-469b-aef8-aa51cec28474 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:12:19.934917 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:12:20.028618 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ff2821-2948-469b-aef8-aa51cec28474 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:12:20.030142 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ff2821-2948-469b-aef8-aa51cec28474 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:12:20.048150 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 667/1326] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:12:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:12:22.362363 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d401004e-55ff-4c52-b7f4-5643f0eac26b None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:12:22.362837 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d401004e-55ff-4c52-b7f4-5643f0eac26b None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:12:22.362951 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d401004e-55ff-4c52-b7f4-5643f0eac26b None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:12:22.364536 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 660/1327] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:12:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:12:22.369629 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5645af-e47e-40e2-9bbe-ae12924ad7eb None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:12:22.370049 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5645af-e47e-40e2-9bbe-ae12924ad7eb None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:12:22.370138 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5645af-e47e-40e2-9bbe-ae12924ad7eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:12:22.435794 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-5c5645af-e47e-40e2-9bbe-ae12924ad7eb None None] MFA Rules not processed for user `720bc045e0bf473dbe5912892ff7b8e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:12:22.436987 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-5c5645af-e47e-40e2-9bbe-ae12924ad7eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:12:22.502461 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:12:22.534271 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 668/1328] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:12:22 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:12:22.587280 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774db874-6ad0-404c-ad27-3f721af72548 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:12:22.587571 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774db874-6ad0-404c-ad27-3f721af72548 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:12:22.587855 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774db874-6ad0-404c-ad27-3f721af72548 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:12:22.589386 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 661/1329] 10.222.0.33 () {54 vars in 964 bytes} [Thu Jun 2 19:12:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:12:22.595265 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bbd156-eff6-49f5-b5de-b549eb26074c None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:12:22.595629 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bbd156-eff6-49f5-b5de-b549eb26074c None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:12:22.595976 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bbd156-eff6-49f5-b5de-b549eb26074c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:12:22.671881 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-b1bbd156-eff6-49f5-b5de-b549eb26074c None None] MFA Rules not processed for user `7417a54046e34e899d7b03f2f2314deb`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:12:22.672989 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-b1bbd156-eff6-49f5-b5de-b549eb26074c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:12:22.753389 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 669/1330] 10.222.0.33 () {58 vars in 1061 bytes} [Thu Jun 2 19:12:22 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:12:22.762550 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774db874-6ad0-404c-ad27-3f721af72548 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:12:22.764103 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-774db874-6ad0-404c-ad27-3f721af72548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:12:22.796319 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:12:22.873243 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774db874-6ad0-404c-ad27-3f721af72548 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:12:22.948461 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147fbc3e-a343-4a73-be70-d4b86d84c1ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:12:22.949027 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147fbc3e-a343-4a73-be70-d4b86d84c1ac service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:12:22.949316 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147fbc3e-a343-4a73-be70-d4b86d84c1ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:12:22.949596 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147fbc3e-a343-4a73-be70-d4b86d84c1ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:12:22.951305 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-147fbc3e-a343-4a73-be70-d4b86d84c1ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:12:23.031134 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:12:23.063552 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147fbc3e-a343-4a73-be70-d4b86d84c1ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:12:23.065128 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147fbc3e-a343-4a73-be70-d4b86d84c1ac service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:12:23.083381 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 662/1331] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:12:22 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:12:26.200474 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7b4f5c-8534-48ae-9586-d4d0e7d33f1a None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:12:26.201577 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7b4f5c-8534-48ae-9586-d4d0e7d33f1a None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:12:26.201577 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7b4f5c-8534-48ae-9586-d4d0e7d33f1a None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:12:26.202989 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 670/1332] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:12:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:12:26.208551 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1d1ab4-ee1f-49b5-8cee-4f09bce8159c None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:12:26.209034 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1d1ab4-ee1f-49b5-8cee-4f09bce8159c None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:12:26.209124 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1d1ab4-ee1f-49b5-8cee-4f09bce8159c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:12:26.280013 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-2a1d1ab4-ee1f-49b5-8cee-4f09bce8159c None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:12:26.281059 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-2a1d1ab4-ee1f-49b5-8cee-4f09bce8159c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:12:26.389156 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 663/1333] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:12:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:12:26.427593 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7b4f5c-8534-48ae-9586-d4d0e7d33f1a None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:12:26.430078 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-ba7b4f5c-8534-48ae-9586-d4d0e7d33f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:12:26.509027 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:12:26.544903 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7b4f5c-8534-48ae-9586-d4d0e7d33f1a None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:12:26.626868 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a77a54a-65a1-42b1-af11-6492923df348 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:12:26.627532 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a77a54a-65a1-42b1-af11-6492923df348 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:12:26.627822 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a77a54a-65a1-42b1-af11-6492923df348 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:12:26.628101 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a77a54a-65a1-42b1-af11-6492923df348 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:12:26.629806 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-4a77a54a-65a1-42b1-af11-6492923df348 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:12:26.745903 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a77a54a-65a1-42b1-af11-6492923df348 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:12:26.746534 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a77a54a-65a1-42b1-af11-6492923df348 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:12:26.757540 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:12:26.766442 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 671/1334] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:12:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:12:29.065231 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d401a960-556f-4239-8b75-7cb9d83d7117 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:12:29.065740 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d401a960-556f-4239-8b75-7cb9d83d7117 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:12:29.065740 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d401a960-556f-4239-8b75-7cb9d83d7117 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:12:29.067112 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 664/1335] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:12:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:12:29.072432 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32468c-2d28-40df-b6b6-bafb3ce4e2d6 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:12:29.072880 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32468c-2d28-40df-b6b6-bafb3ce4e2d6 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:12:29.073001 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32468c-2d28-40df-b6b6-bafb3ce4e2d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:12:29.138969 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-0e32468c-2d28-40df-b6b6-bafb3ce4e2d6 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:12:29.140201 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-0e32468c-2d28-40df-b6b6-bafb3ce4e2d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:12:29.234374 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 672/1336] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:12:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:12:29.275013 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d401a960-556f-4239-8b75-7cb9d83d7117 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:12:29.276554 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-d401a960-556f-4239-8b75-7cb9d83d7117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:12:29.316257 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:12:29.380619 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d401a960-556f-4239-8b75-7cb9d83d7117 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:12:29.453512 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31f422c-6da8-490e-9a74-382b9e3e6959 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:12:29.454151 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31f422c-6da8-490e-9a74-382b9e3e6959 service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:12:29.454434 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31f422c-6da8-490e-9a74-382b9e3e6959 service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:12:29.454707 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31f422c-6da8-490e-9a74-382b9e3e6959 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:12:29.456364 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-b31f422c-6da8-490e-9a74-382b9e3e6959 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:12:29.540399 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:12:29.565220 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31f422c-6da8-490e-9a74-382b9e3e6959 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:12:29.566724 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31f422c-6da8-490e-9a74-382b9e3e6959 service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:12:29.585477 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 665/1337] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:12:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:12:32.505019 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c20d79-d1c5-41a5-bd60-e2c299c22f7e None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:12:32.505564 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c20d79-d1c5-41a5-bd60-e2c299c22f7e None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:12:32.505564 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c20d79-d1c5-41a5-bd60-e2c299c22f7e None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:12:32.507017 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 673/1338] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:12:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:12:32.512320 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517a74c9-ae92-45b4-9081-6b30f9aab7b3 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:12:32.512782 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517a74c9-ae92-45b4-9081-6b30f9aab7b3 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:12:32.512889 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517a74c9-ae92-45b4-9081-6b30f9aab7b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:12:32.581185 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-517a74c9-ae92-45b4-9081-6b30f9aab7b3 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:12:32.582377 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-517a74c9-ae92-45b4-9081-6b30f9aab7b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:12:32.685974 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 666/1339] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:12:32 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:12:32.721168 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c20d79-d1c5-41a5-bd60-e2c299c22f7e None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:12:32.723342 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-26c20d79-d1c5-41a5-bd60-e2c299c22f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:12:32.805177 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:12:32.821215 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c20d79-d1c5-41a5-bd60-e2c299c22f7e None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:12:32.823138 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b4d251-5a00-48f1-83cb-84bdb7099be1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:12:32.823755 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b4d251-5a00-48f1-83cb-84bdb7099be1 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:12:32.824047 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b4d251-5a00-48f1-83cb-84bdb7099be1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:12:32.824323 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b4d251-5a00-48f1-83cb-84bdb7099be1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:12:32.833473 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b4d251-5a00-48f1-83cb-84bdb7099be1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:12:32.834940 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b4d251-5a00-48f1-83cb-84bdb7099be1 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:12:32.844685 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 674/1340] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:12:32 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:12:37.366560 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb200d2f-780d-40d7-aab1-ea1c3912e1d5 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:12:37.367152 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb200d2f-780d-40d7-aab1-ea1c3912e1d5 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:12:37.367152 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb200d2f-780d-40d7-aab1-ea1c3912e1d5 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:12:37.368591 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 667/1341] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:12:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:12:37.373850 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4902ee86-7cc5-46a9-913d-a36a28a8691b None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:12:37.374336 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4902ee86-7cc5-46a9-913d-a36a28a8691b None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:12:37.374448 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4902ee86-7cc5-46a9-913d-a36a28a8691b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:12:37.404205 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-4902ee86-7cc5-46a9-913d-a36a28a8691b None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:12:37.405413 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-4902ee86-7cc5-46a9-913d-a36a28a8691b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:12:37.494555 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 675/1342] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:12:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:12:37.533005 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb200d2f-780d-40d7-aab1-ea1c3912e1d5 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:12:37.543874 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb200d2f-780d-40d7-aab1-ea1c3912e1d5 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:12:37.548079 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d662db9-10fe-44b4-a648-e5c798f6850f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:12:37.548816 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d662db9-10fe-44b4-a648-e5c798f6850f service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:12:37.549188 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d662db9-10fe-44b4-a648-e5c798f6850f service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:12:37.549532 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d662db9-10fe-44b4-a648-e5c798f6850f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:12:37.558861 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d662db9-10fe-44b4-a648-e5c798f6850f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:12:37.560489 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d662db9-10fe-44b4-a648-e5c798f6850f service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:12:37.569995 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 668/1343] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:12:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:12:40.847827 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a439cedc-68d9-4a43-8780-041524e7c73f None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:12:40.848359 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a439cedc-68d9-4a43-8780-041524e7c73f None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:12:40.848359 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a439cedc-68d9-4a43-8780-041524e7c73f None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:12:40.849668 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 676/1344] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:12:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:12:40.854768 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d72e2-1269-4169-9e00-c3fa43527178 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:12:40.855305 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d72e2-1269-4169-9e00-c3fa43527178 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:12:40.855455 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d72e2-1269-4169-9e00-c3fa43527178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:12:40.873315 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-337d72e2-1269-4169-9e00-c3fa43527178 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:12:40.874516 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-337d72e2-1269-4169-9e00-c3fa43527178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:12:40.882621 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 669/1345] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:12:40 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:12:40.919673 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a439cedc-68d9-4a43-8780-041524e7c73f None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:12:40.930119 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a439cedc-68d9-4a43-8780-041524e7c73f None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:12:40.933833 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f21e935-1f38-4175-8f90-a8150782c631 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:12:40.934469 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f21e935-1f38-4175-8f90-a8150782c631 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:12:40.934852 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f21e935-1f38-4175-8f90-a8150782c631 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:12:40.935091 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f21e935-1f38-4175-8f90-a8150782c631 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:12:40.944244 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f21e935-1f38-4175-8f90-a8150782c631 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:12:40.945674 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f21e935-1f38-4175-8f90-a8150782c631 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:12:40.955742 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 677/1346] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:12:40 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:12:45.448721 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279ff60-9c9e-4d7c-a6d4-6b72d735dc78 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:12:45.449226 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279ff60-9c9e-4d7c-a6d4-6b72d735dc78 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:12:45.449226 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279ff60-9c9e-4d7c-a6d4-6b72d735dc78 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:12:45.451128 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 670/1347] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:12:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:12:45.456274 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d44171e-bbac-4dfd-84e5-f841cdd32752 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:12:45.456688 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d44171e-bbac-4dfd-84e5-f841cdd32752 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:12:45.456830 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d44171e-bbac-4dfd-84e5-f841cdd32752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:12:45.475506 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-6d44171e-bbac-4dfd-84e5-f841cdd32752 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:12:45.476705 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-6d44171e-bbac-4dfd-84e5-f841cdd32752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:12:45.484991 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 678/1348] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:12:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:12:45.523299 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3279ff60-9c9e-4d7c-a6d4-6b72d735dc78 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:12:45.534155 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3279ff60-9c9e-4d7c-a6d4-6b72d735dc78 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:12:45.537628 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ec89c-77bd-46a6-a651-f97cb119889b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:12:45.538284 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ec89c-77bd-46a6-a651-f97cb119889b service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:12:45.538561 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ec89c-77bd-46a6-a651-f97cb119889b service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:12:45.538869 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ec89c-77bd-46a6-a651-f97cb119889b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:12:45.547701 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3ec89c-77bd-46a6-a651-f97cb119889b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:12:45.549192 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3ec89c-77bd-46a6-a651-f97cb119889b service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:12:45.558467 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 671/1349] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:12:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:12:51.039501 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce7629-c2e0-40c5-aabe-b59ed613e9ee None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:12:51.040057 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce7629-c2e0-40c5-aabe-b59ed613e9ee None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:12:51.040429 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce7629-c2e0-40c5-aabe-b59ed613e9ee None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:12:51.042525 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 679/1350] 10.222.0.33 () {54 vars in 921 bytes} [Thu Jun 2 19:12:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:12:51.048299 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fe1639-a8d8-4820-943c-1fdd40afaeb3 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:12:51.048736 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fe1639-a8d8-4820-943c-1fdd40afaeb3 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:12:51.049954 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fe1639-a8d8-4820-943c-1fdd40afaeb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:12:51.075182 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:12:51.144804 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-85fe1639-a8d8-4820-943c-1fdd40afaeb3 None None] MFA Rules not processed for user `73ae837e99764bebbe7a6e06ac9049e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:12:51.146429 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-85fe1639-a8d8-4820-943c-1fdd40afaeb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:12:51.281184 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 672/1351] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Jun 2 19:12:51 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:12:55.470079 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=57 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:12:55.814619 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a01200-c463-410e-aede-3131c285d518 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:12:55.815015 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a01200-c463-410e-aede-3131c285d518 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:12:55.815111 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a01200-c463-410e-aede-3131c285d518 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:12:55.816666 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 680/1352] 10.222.0.33 () {54 vars in 921 bytes} [Thu Jun 2 19:12:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 02 19:12:55.821905 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ba59e-96b0-4944-9b7b-d1bee04eb8b6 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:12:55.822616 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ba59e-96b0-4944-9b7b-d1bee04eb8b6 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:12:55.822616 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ba59e-96b0-4944-9b7b-d1bee04eb8b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:12:55.842600 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-fb5ba59e-96b0-4944-9b7b-d1bee04eb8b6 None None] MFA Rules not processed for user `73ae837e99764bebbe7a6e06ac9049e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:12:55.843839 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-fb5ba59e-96b0-4944-9b7b-d1bee04eb8b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:12:55.853449 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 673/1353] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Jun 2 19:12:55 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:12:57.078602 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af735925-9224-4f23-aec9-27e2c5ea58d1 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:12:57.079208 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af735925-9224-4f23-aec9-27e2c5ea58d1 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:12:57.079208 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af735925-9224-4f23-aec9-27e2c5ea58d1 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:12:57.080816 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 681/1354] 10.222.0.33 () {54 vars in 921 bytes} [Thu Jun 2 19:12:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:12:57.086114 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4e003b-078c-49e8-a27f-9487057ebea2 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:12:57.086772 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4e003b-078c-49e8-a27f-9487057ebea2 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:12:57.086772 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4e003b-078c-49e8-a27f-9487057ebea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:12:57.105021 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-1a4e003b-078c-49e8-a27f-9487057ebea2 None None] MFA Rules not processed for user `73ae837e99764bebbe7a6e06ac9049e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:12:57.106218 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-1a4e003b-078c-49e8-a27f-9487057ebea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:12:57.115924 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 674/1355] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Jun 2 19:12:57 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:13:01.177688 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7f8b9e-36d8-4d87-9cab-0c11f58b0667 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:01.178246 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7f8b9e-36d8-4d87-9cab-0c11f58b0667 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:01.178246 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7f8b9e-36d8-4d87-9cab-0c11f58b0667 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:01.179556 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 682/1356] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:13:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:13:01.185430 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e7f5de-9d95-4fb5-a112-e579dd65021c None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:01.185900 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e7f5de-9d95-4fb5-a112-e579dd65021c None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:01.186107 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e7f5de-9d95-4fb5-a112-e579dd65021c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:01.206662 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-99e7f5de-9d95-4fb5-a112-e579dd65021c None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:13:01.208271 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-99e7f5de-9d95-4fb5-a112-e579dd65021c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:13:01.218110 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 675/1357] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:13:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:13:01.271864 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1531a4-a298-4a6c-97d4-448527a500cd None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:01.272420 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1531a4-a298-4a6c-97d4-448527a500cd None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:01.272946 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1531a4-a298-4a6c-97d4-448527a500cd None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:01.274786 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 683/1358] 10.222.0.33 () {54 vars in 961 bytes} [Thu Jun 2 19:13:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:13:01.285049 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd033718-9bb2-4607-8a78-d1fcc3bc072f None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:01.285346 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd033718-9bb2-4607-8a78-d1fcc3bc072f None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:01.285640 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd033718-9bb2-4607-8a78-d1fcc3bc072f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:01.344139 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-dd033718-9bb2-4607-8a78-d1fcc3bc072f None None] MFA Rules not processed for user `70664d5cbdee47ee89c120761939cc75`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:13:01.345341 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-dd033718-9bb2-4607-8a78-d1fcc3bc072f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:13:01.412671 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:13:01.435638 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 676/1359] 10.222.0.33 () {58 vars in 1058 bytes} [Thu Jun 2 19:13:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:13:01.443305 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1531a4-a298-4a6c-97d4-448527a500cd None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:13:01.458042 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1531a4-a298-4a6c-97d4-448527a500cd None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:13:01.463545 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c04325a-4ce6-4801-a5f7-b295bd9da077 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:13:01.465353 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c04325a-4ce6-4801-a5f7-b295bd9da077 service nova] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:01.465982 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c04325a-4ce6-4801-a5f7-b295bd9da077 service nova] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:01.466260 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c04325a-4ce6-4801-a5f7-b295bd9da077 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:01.474712 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c04325a-4ce6-4801-a5f7-b295bd9da077 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:13:01.476322 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c04325a-4ce6-4801-a5f7-b295bd9da077 service nova] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:13:01.486247 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 684/1360] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:13:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:13:03.963517 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6c5f6b-80b1-48a8-8b88-3717a2c2cfe6 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:03.964083 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6c5f6b-80b1-48a8-8b88-3717a2c2cfe6 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:03.964083 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6c5f6b-80b1-48a8-8b88-3717a2c2cfe6 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:03.965595 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 677/1361] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:13:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:13:03.970997 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864dd5ee-dc9c-4c5d-a122-1c73e67a0289 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:03.971870 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864dd5ee-dc9c-4c5d-a122-1c73e67a0289 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:03.972159 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864dd5ee-dc9c-4c5d-a122-1c73e67a0289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:03.991184 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-864dd5ee-dc9c-4c5d-a122-1c73e67a0289 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:13:03.992373 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-864dd5ee-dc9c-4c5d-a122-1c73e67a0289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:13:04.005661 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 685/1362] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:13:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:13:04.046092 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6c5f6b-80b1-48a8-8b88-3717a2c2cfe6 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:13:04.061632 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6c5f6b-80b1-48a8-8b88-3717a2c2cfe6 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:13:04.067416 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66109c80-999a-4058-bea9-9967b3193c0c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:13:04.068241 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66109c80-999a-4058-bea9-9967b3193c0c service nova] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:04.068701 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66109c80-999a-4058-bea9-9967b3193c0c service nova] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:04.069122 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66109c80-999a-4058-bea9-9967b3193c0c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:04.084033 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66109c80-999a-4058-bea9-9967b3193c0c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:13:04.085860 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66109c80-999a-4058-bea9-9967b3193c0c service nova] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:13:04.097825 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 678/1363] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:13:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:13:06.961453 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24351989-9f66-46ba-8287-29af9c252720 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:06.961960 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24351989-9f66-46ba-8287-29af9c252720 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:06.962121 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24351989-9f66-46ba-8287-29af9c252720 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:06.964789 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 686/1364] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:13:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:13:06.972620 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0996e0-c8dd-4ed4-a705-1836414cf32a None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:06.972620 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0996e0-c8dd-4ed4-a705-1836414cf32a None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:06.972620 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0996e0-c8dd-4ed4-a705-1836414cf32a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:06.991139 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-ae0996e0-c8dd-4ed4-a705-1836414cf32a None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:13:06.992430 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-ae0996e0-c8dd-4ed4-a705-1836414cf32a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:13:07.001129 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 679/1365] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:13:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:13:07.048111 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c55ccb-ec6c-4caf-8af0-3e00e2447491 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:07.049160 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c55ccb-ec6c-4caf-8af0-3e00e2447491 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:07.049575 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c55ccb-ec6c-4caf-8af0-3e00e2447491 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:07.051103 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 687/1366] 10.222.0.33 () {54 vars in 961 bytes} [Thu Jun 2 19:13:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:13:07.056479 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7891c026-f92a-4a63-b28e-e7108fb228fc None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:07.056769 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7891c026-f92a-4a63-b28e-e7108fb228fc None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:07.057048 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7891c026-f92a-4a63-b28e-e7108fb228fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:07.077887 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-7891c026-f92a-4a63-b28e-e7108fb228fc None None] MFA Rules not processed for user `70664d5cbdee47ee89c120761939cc75`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:13:07.079219 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-7891c026-f92a-4a63-b28e-e7108fb228fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:13:07.089584 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 680/1367] 10.222.0.33 () {58 vars in 1058 bytes} [Thu Jun 2 19:13:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:13:07.097975 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c55ccb-ec6c-4caf-8af0-3e00e2447491 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:13:07.113269 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c55ccb-ec6c-4caf-8af0-3e00e2447491 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:13:07.119478 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df182b4-1562-4cf8-888d-0b371930cb1a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:13:07.120209 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df182b4-1562-4cf8-888d-0b371930cb1a service nova] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:07.120605 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df182b4-1562-4cf8-888d-0b371930cb1a service nova] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:07.120960 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df182b4-1562-4cf8-888d-0b371930cb1a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:07.130630 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df182b4-1562-4cf8-888d-0b371930cb1a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:13:07.132351 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df182b4-1562-4cf8-888d-0b371930cb1a service nova] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:13:07.143606 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 688/1368] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:13:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:13:09.645677 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1109fe-ce91-48e9-8dda-78e8fdc6c1cf None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:09.646279 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1109fe-ce91-48e9-8dda-78e8fdc6c1cf None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:09.646279 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1109fe-ce91-48e9-8dda-78e8fdc6c1cf None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:09.650805 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 681/1369] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:13:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:13:09.653548 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee18b217-2835-456b-aac3-11747d2dd9e0 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:09.654013 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee18b217-2835-456b-aac3-11747d2dd9e0 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:09.654104 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee18b217-2835-456b-aac3-11747d2dd9e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:09.672503 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-ee18b217-2835-456b-aac3-11747d2dd9e0 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:13:09.673714 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-ee18b217-2835-456b-aac3-11747d2dd9e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:13:09.681862 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 689/1370] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:13:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:13:09.728050 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1109fe-ce91-48e9-8dda-78e8fdc6c1cf None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:13:09.745614 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1109fe-ce91-48e9-8dda-78e8fdc6c1cf None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:13:09.752950 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bdd488-57d8-4b86-b288-66869107ba10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:13:09.754086 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bdd488-57d8-4b86-b288-66869107ba10 service nova] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:09.754692 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bdd488-57d8-4b86-b288-66869107ba10 service nova] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:09.755166 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bdd488-57d8-4b86-b288-66869107ba10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:09.765924 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bdd488-57d8-4b86-b288-66869107ba10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:13:09.767579 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bdd488-57d8-4b86-b288-66869107ba10 service nova] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:13:09.780496 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 682/1371] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:13:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:13:09.795756 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74725703-a582-4678-a544-a669c6d12bf7 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:09.796126 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74725703-a582-4678-a544-a669c6d12bf7 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:09.796503 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74725703-a582-4678-a544-a669c6d12bf7 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:09.798010 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 690/1372] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:13:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:13:09.803433 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97bb948-17d4-4b5b-961f-eefa31dea9e7 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:09.803735 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97bb948-17d4-4b5b-961f-eefa31dea9e7 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:09.804005 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97bb948-17d4-4b5b-961f-eefa31dea9e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:09.822455 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-e97bb948-17d4-4b5b-961f-eefa31dea9e7 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:13:09.823771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-e97bb948-17d4-4b5b-961f-eefa31dea9e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:13:09.832686 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 683/1373] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:13:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:13:10.153102 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e5dc3-be08-45db-997c-59d838887404 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:10.153102 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e5dc3-be08-45db-997c-59d838887404 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:10.153102 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e5dc3-be08-45db-997c-59d838887404 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:10.153102 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 691/1374] 10.222.0.33 () {54 vars in 963 bytes} [Thu Jun 2 19:13:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:13:10.158998 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac23804-d4d0-4315-8cd5-87ba2d8838e5 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:10.159344 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac23804-d4d0-4315-8cd5-87ba2d8838e5 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:10.159605 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac23804-d4d0-4315-8cd5-87ba2d8838e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:10.217974 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-dac23804-d4d0-4315-8cd5-87ba2d8838e5 None None] MFA Rules not processed for user `091860b2577f49c5af61c75803a90c0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:13:10.219376 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-dac23804-d4d0-4315-8cd5-87ba2d8838e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:13:10.286787 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 684/1375] 10.222.0.33 () {58 vars in 1060 bytes} [Thu Jun 2 19:13:10 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 19:13:10.295477 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e5dc3-be08-45db-997c-59d838887404 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:13:10.307637 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e5dc3-be08-45db-997c-59d838887404 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:13:10.311427 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffc4a4b-411d-4cf9-b3ff-d2ebb48b4be8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:13:10.312157 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffc4a4b-411d-4cf9-b3ff-d2ebb48b4be8 service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:10.312511 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffc4a4b-411d-4cf9-b3ff-d2ebb48b4be8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:10.312970 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffc4a4b-411d-4cf9-b3ff-d2ebb48b4be8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:10.323377 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffc4a4b-411d-4cf9-b3ff-d2ebb48b4be8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:13:10.324868 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffc4a4b-411d-4cf9-b3ff-d2ebb48b4be8 service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:13:10.335696 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 692/1376] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:13:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:13:12.219654 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80afbf9e-e028-4ed3-b907-3a6076735fc6 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:12.220138 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80afbf9e-e028-4ed3-b907-3a6076735fc6 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:12.220233 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80afbf9e-e028-4ed3-b907-3a6076735fc6 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:12.221718 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 685/1377] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:13:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:13:12.228000 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe40f4ab-7030-4dcc-9d75-91ad3be9f1fb None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:12.228579 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe40f4ab-7030-4dcc-9d75-91ad3be9f1fb None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:12.228703 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe40f4ab-7030-4dcc-9d75-91ad3be9f1fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:12.248722 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-fe40f4ab-7030-4dcc-9d75-91ad3be9f1fb None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:13:12.250111 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-fe40f4ab-7030-4dcc-9d75-91ad3be9f1fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:13:12.258703 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 693/1378] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:13:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:13:12.305799 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80afbf9e-e028-4ed3-b907-3a6076735fc6 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:13:12.330391 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80afbf9e-e028-4ed3-b907-3a6076735fc6 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:13:12.336043 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac39d11-af5c-45a6-816e-58388ddd6c34 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:13:12.336715 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac39d11-af5c-45a6-816e-58388ddd6c34 service nova] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:12.336997 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac39d11-af5c-45a6-816e-58388ddd6c34 service nova] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:12.337273 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac39d11-af5c-45a6-816e-58388ddd6c34 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:12.348649 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac39d11-af5c-45a6-816e-58388ddd6c34 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:13:12.350278 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac39d11-af5c-45a6-816e-58388ddd6c34 service nova] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:13:12.362788 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 686/1379] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:13:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:13:14.910547 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0479493-a0d1-44a0-835c-5ede4e833dce None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:14.911150 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0479493-a0d1-44a0-835c-5ede4e833dce None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:14.911443 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0479493-a0d1-44a0-835c-5ede4e833dce None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:14.913139 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 694/1380] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:13:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:13:14.918566 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3434c020-52d5-4a35-a07e-877158a6e110 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:14.919207 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3434c020-52d5-4a35-a07e-877158a6e110 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:14.919207 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3434c020-52d5-4a35-a07e-877158a6e110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:14.939841 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-3434c020-52d5-4a35-a07e-877158a6e110 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:13:14.941636 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-3434c020-52d5-4a35-a07e-877158a6e110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:13:14.953356 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 687/1381] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:13:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:13:15.001206 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0479493-a0d1-44a0-835c-5ede4e833dce None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:13:15.016528 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0479493-a0d1-44a0-835c-5ede4e833dce None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:13:15.022049 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0ee243-ac56-4f6c-9b82-d35af12505cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:13:15.022851 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ee243-ac56-4f6c-9b82-d35af12505cf service nova] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:15.023454 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ee243-ac56-4f6c-9b82-d35af12505cf service nova] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:15.023454 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ee243-ac56-4f6c-9b82-d35af12505cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:15.034924 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0ee243-ac56-4f6c-9b82-d35af12505cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:13:15.036534 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0ee243-ac56-4f6c-9b82-d35af12505cf service nova] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:13:15.047086 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 695/1382] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:13:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:13:17.697159 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010bf48-11cb-42b3-8943-868bcbfccc0d None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:17.697743 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010bf48-11cb-42b3-8943-868bcbfccc0d None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:17.697743 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010bf48-11cb-42b3-8943-868bcbfccc0d None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:17.699283 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 688/1383] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:13:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:13:17.706941 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef38d8c-9696-4019-9632-7229eb721bcc None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:17.706941 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef38d8c-9696-4019-9632-7229eb721bcc None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:17.706941 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef38d8c-9696-4019-9632-7229eb721bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:17.725106 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-1ef38d8c-9696-4019-9632-7229eb721bcc None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:13:17.726632 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-1ef38d8c-9696-4019-9632-7229eb721bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:13:17.736290 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 696/1384] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:13:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:13:17.779552 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c010bf48-11cb-42b3-8943-868bcbfccc0d None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:13:17.796268 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c010bf48-11cb-42b3-8943-868bcbfccc0d None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:13:17.802531 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f719c652-e81e-4821-adac-3e367f289684 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:13:17.803525 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f719c652-e81e-4821-adac-3e367f289684 service nova] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:17.803937 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f719c652-e81e-4821-adac-3e367f289684 service nova] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:17.804322 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f719c652-e81e-4821-adac-3e367f289684 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:17.814420 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f719c652-e81e-4821-adac-3e367f289684 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:13:17.816207 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f719c652-e81e-4821-adac-3e367f289684 service nova] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:13:17.829042 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 689/1385] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:13:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:13:20.274395 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c51bfd-53b2-4462-8030-a306f6a8cb0e None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:20.275133 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c51bfd-53b2-4462-8030-a306f6a8cb0e None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:20.275133 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c51bfd-53b2-4462-8030-a306f6a8cb0e None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:20.276532 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 697/1386] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:13:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:13:20.283435 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078682c5-d1a0-44f9-bfff-c9db6129e9b9 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:20.283918 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078682c5-d1a0-44f9-bfff-c9db6129e9b9 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:20.284171 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078682c5-d1a0-44f9-bfff-c9db6129e9b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:20.306641 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-078682c5-d1a0-44f9-bfff-c9db6129e9b9 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:13:20.308364 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-078682c5-d1a0-44f9-bfff-c9db6129e9b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:13:20.318263 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 690/1387] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:13:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:13:20.375876 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c51bfd-53b2-4462-8030-a306f6a8cb0e None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:13:20.402933 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c51bfd-53b2-4462-8030-a306f6a8cb0e None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:13:20.409042 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14972f2f-c307-42c4-b4f9-53e8ddcebe99 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:13:20.409705 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14972f2f-c307-42c4-b4f9-53e8ddcebe99 service nova] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:20.409987 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14972f2f-c307-42c4-b4f9-53e8ddcebe99 service nova] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:20.410262 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14972f2f-c307-42c4-b4f9-53e8ddcebe99 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:20.419901 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14972f2f-c307-42c4-b4f9-53e8ddcebe99 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:13:20.421528 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14972f2f-c307-42c4-b4f9-53e8ddcebe99 service nova] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:13:20.431698 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 698/1388] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:13:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:13:23.183635 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b632bbe-aa02-4fa6-ba83-bddd00fb3514 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:23.184736 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b632bbe-aa02-4fa6-ba83-bddd00fb3514 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:23.184736 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b632bbe-aa02-4fa6-ba83-bddd00fb3514 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:23.186084 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 691/1389] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:13:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:13:23.192077 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba0fc7-7656-4208-910c-69def0d81e6b None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:23.192591 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba0fc7-7656-4208-910c-69def0d81e6b None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:23.192756 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba0fc7-7656-4208-910c-69def0d81e6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:23.211570 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-caba0fc7-7656-4208-910c-69def0d81e6b None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:13:23.212779 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-caba0fc7-7656-4208-910c-69def0d81e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:13:23.220865 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 699/1390] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:13:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:13:23.269108 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b632bbe-aa02-4fa6-ba83-bddd00fb3514 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:13:23.288563 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b632bbe-aa02-4fa6-ba83-bddd00fb3514 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:13:23.294634 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8c0e86-0c9e-4111-b497-1fdd0bd119aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:13:23.295500 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c0e86-0c9e-4111-b497-1fdd0bd119aa service nova] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:23.295890 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c0e86-0c9e-4111-b497-1fdd0bd119aa service nova] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:23.296278 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c0e86-0c9e-4111-b497-1fdd0bd119aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:23.307414 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8c0e86-0c9e-4111-b497-1fdd0bd119aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:13:23.309187 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8c0e86-0c9e-4111-b497-1fdd0bd119aa service nova] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:13:23.322141 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 692/1391] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:13:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 02 19:13:25.989604 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92daed80-ee79-44a6-bfa1-9e0c862e182f None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:25.990222 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92daed80-ee79-44a6-bfa1-9e0c862e182f None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:25.990407 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92daed80-ee79-44a6-bfa1-9e0c862e182f None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:25.991799 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 700/1392] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:13:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:13:25.997811 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5ef8e3-2903-49af-bb0a-794f82f7b0ee None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:26.000041 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5ef8e3-2903-49af-bb0a-794f82f7b0ee None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:26.000041 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5ef8e3-2903-49af-bb0a-794f82f7b0ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:26.016929 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-8e5ef8e3-2903-49af-bb0a-794f82f7b0ee None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:13:26.018182 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-8e5ef8e3-2903-49af-bb0a-794f82f7b0ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:13:26.026227 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 693/1393] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:13:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:13:26.071448 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92daed80-ee79-44a6-bfa1-9e0c862e182f None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:13:26.080099 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=46 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:13:26.086878 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92daed80-ee79-44a6-bfa1-9e0c862e182f None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:13:26.092478 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4231eab0-97cd-4906-8a6d-cb42dc1b18e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:13:26.093123 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4231eab0-97cd-4906-8a6d-cb42dc1b18e8 service nova] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:26.093402 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4231eab0-97cd-4906-8a6d-cb42dc1b18e8 service nova] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:26.093679 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4231eab0-97cd-4906-8a6d-cb42dc1b18e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:26.102140 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4231eab0-97cd-4906-8a6d-cb42dc1b18e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:13:26.103752 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4231eab0-97cd-4906-8a6d-cb42dc1b18e8 service nova] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:13:26.113705 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 701/1394] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:13:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 02 19:13:27.045694 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0fb425-3e70-4756-a644-04eef096f58c None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:27.046255 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0fb425-3e70-4756-a644-04eef096f58c None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:27.046255 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0fb425-3e70-4756-a644-04eef096f58c None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:27.050698 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 694/1395] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:13:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:13:27.054183 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02bcf9d-5c09-475c-8b2e-b85f8c33b5ec None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:27.054684 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02bcf9d-5c09-475c-8b2e-b85f8c33b5ec None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:27.054841 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02bcf9d-5c09-475c-8b2e-b85f8c33b5ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:27.072923 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-c02bcf9d-5c09-475c-8b2e-b85f8c33b5ec None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:13:27.074211 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-c02bcf9d-5c09-475c-8b2e-b85f8c33b5ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:13:27.089243 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 702/1396] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:13:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 19:13:27.148369 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce61416d-8499-4f3a-bf20-a97893630db4 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:27.148664 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce61416d-8499-4f3a-bf20-a97893630db4 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:27.148944 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce61416d-8499-4f3a-bf20-a97893630db4 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:27.150303 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 695/1397] 10.222.0.33 () {54 vars in 962 bytes} [Thu Jun 2 19:13:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:13:27.155967 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e30e961-a8d8-4362-ae71-ed34ba60eb17 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:27.156801 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e30e961-a8d8-4362-ae71-ed34ba60eb17 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:27.157648 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e30e961-a8d8-4362-ae71-ed34ba60eb17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:27.209327 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-3e30e961-a8d8-4362-ae71-ed34ba60eb17 None None] MFA Rules not processed for user `e07892c0aa21449b94a1cc78e3b8a653`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:13:27.210442 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-3e30e961-a8d8-4362-ae71-ed34ba60eb17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:13:27.281794 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 703/1398] 10.222.0.33 () {58 vars in 1059 bytes} [Thu Jun 2 19:13:27 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:13:27.289676 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce61416d-8499-4f3a-bf20-a97893630db4 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:13:27.302443 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce61416d-8499-4f3a-bf20-a97893630db4 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:13:27.307665 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437c3a7f-370c-4ae4-96a2-baa64e851889 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07892c0aa21449b94a1cc78e3b8a653', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:13:27.308297 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c3a7f-370c-4ae4-96a2-baa64e851889 service glance] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:27.308586 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c3a7f-370c-4ae4-96a2-baa64e851889 service glance] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:27.308856 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c3a7f-370c-4ae4-96a2-baa64e851889 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:27.318451 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437c3a7f-370c-4ae4-96a2-baa64e851889 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:13:27.319929 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437c3a7f-370c-4ae4-96a2-baa64e851889 service glance] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:13:27.330738 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 696/1399] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:13:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:13:28.032000 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4a035b-7468-438f-a9d6-3ba42ecf9e0c None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:28.032298 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4a035b-7468-438f-a9d6-3ba42ecf9e0c None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:28.032582 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4a035b-7468-438f-a9d6-3ba42ecf9e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:28.087433 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-1f4a035b-7468-438f-a9d6-3ba42ecf9e0c None None] MFA Rules not processed for user `04f64813f09348eb91cb7bb2869f7612`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:13:28.088646 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-1f4a035b-7468-438f-a9d6-3ba42ecf9e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:13:28.176013 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 704/1400] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Jun 2 19:13:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:13:28.222822 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0425814a-87c4-402f-bb37-ced9b8bae8b7 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:28.223352 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0425814a-87c4-402f-bb37-ced9b8bae8b7 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:28.223495 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0425814a-87c4-402f-bb37-ced9b8bae8b7 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:28.224777 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 697/1401] 10.222.0.33 () {54 vars in 942 bytes} [Thu Jun 2 19:13:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:13:28.231819 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8624e88-9f7d-4426-a05e-609570293a14 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:28.232538 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8624e88-9f7d-4426-a05e-609570293a14 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:28.233091 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8624e88-9f7d-4426-a05e-609570293a14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:28.291762 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:13:28.295727 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-e8624e88-9f7d-4426-a05e-609570293a14 None None] MFA Rules not processed for user `2a59e1d67ff94186a3ebcfcb49938a87`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:13:28.297053 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-e8624e88-9f7d-4426-a05e-609570293a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:13:28.386339 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 705/1402] 10.222.0.33 () {58 vars in 1039 bytes} [Thu Jun 2 19:13:28 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:13:28.397014 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0425814a-87c4-402f-bb37-ced9b8bae8b7 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:13:28.412789 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0425814a-87c4-402f-bb37-ced9b8bae8b7 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:13:28.418205 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9199f412-6332-4575-ba93-1322a3bac5ee service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a59e1d67ff94186a3ebcfcb49938a87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:13:28.418945 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9199f412-6332-4575-ba93-1322a3bac5ee service swift] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:28.419231 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9199f412-6332-4575-ba93-1322a3bac5ee service swift] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:28.419515 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9199f412-6332-4575-ba93-1322a3bac5ee service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:28.427407 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:13:28.433923 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9199f412-6332-4575-ba93-1322a3bac5ee service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:13:28.436133 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9199f412-6332-4575-ba93-1322a3bac5ee service swift] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:13:28.445426 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 698/1403] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 2 19:13:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 02 19:13:28.583591 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468085db-79ed-4117-b705-a2768ed1f148 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:28.583883 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468085db-79ed-4117-b705-a2768ed1f148 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:28.584166 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468085db-79ed-4117-b705-a2768ed1f148 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:28.585550 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 706/1404] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:13:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:13:28.590837 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae7ffd-87f3-42e4-98d8-9df8ac462696 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:28.591128 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae7ffd-87f3-42e4-98d8-9df8ac462696 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:28.591551 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae7ffd-87f3-42e4-98d8-9df8ac462696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:28.609330 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-f8ae7ffd-87f3-42e4-98d8-9df8ac462696 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:13:28.610813 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-f8ae7ffd-87f3-42e4-98d8-9df8ac462696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:13:28.618536 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 699/1405] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:13:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:13:28.662451 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468085db-79ed-4117-b705-a2768ed1f148 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:13:28.676991 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468085db-79ed-4117-b705-a2768ed1f148 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:13:28.682834 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f19726-4ea8-4b94-b022-46152243d750 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:13:28.683074 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f19726-4ea8-4b94-b022-46152243d750 service nova] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:28.683281 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f19726-4ea8-4b94-b022-46152243d750 service nova] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:28.683551 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f19726-4ea8-4b94-b022-46152243d750 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:28.693147 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f19726-4ea8-4b94-b022-46152243d750 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:13:28.694581 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f19726-4ea8-4b94-b022-46152243d750 service nova] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:13:28.704785 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 707/1406] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:13:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:13:31.272404 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c7c5fb-de21-4169-94fa-b132e8b5200f None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:31.273353 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c7c5fb-de21-4169-94fa-b132e8b5200f None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:31.273764 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c7c5fb-de21-4169-94fa-b132e8b5200f None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:31.275630 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 700/1407] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:13:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:13:31.282465 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbfba3e-a6e7-4327-8d41-26cb72a81bfe None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:31.283544 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbfba3e-a6e7-4327-8d41-26cb72a81bfe None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:31.284042 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbfba3e-a6e7-4327-8d41-26cb72a81bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:31.303338 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-9fbfba3e-a6e7-4327-8d41-26cb72a81bfe None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:13:31.304915 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-9fbfba3e-a6e7-4327-8d41-26cb72a81bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:13:31.319243 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 708/1408] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:13:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:13:31.374277 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c7c5fb-de21-4169-94fa-b132e8b5200f None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:13:31.390433 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c7c5fb-de21-4169-94fa-b132e8b5200f None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:13:31.402543 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3e45fe-bcbf-4580-bfed-cf6b13cc37f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:13:31.404081 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e45fe-bcbf-4580-bfed-cf6b13cc37f1 service nova] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:31.404379 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e45fe-bcbf-4580-bfed-cf6b13cc37f1 service nova] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:31.404654 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e45fe-bcbf-4580-bfed-cf6b13cc37f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:31.414525 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3e45fe-bcbf-4580-bfed-cf6b13cc37f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:13:31.416361 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3e45fe-bcbf-4580-bfed-cf6b13cc37f1 service nova] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:13:31.427945 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 701/1409] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:13:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:13:41.297592 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=39 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:13:41.422604 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=12 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:13:52.927386 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f4fef-dae7-4f59-9c05-5f54ff64f60f None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:52.928313 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f4fef-dae7-4f59-9c05-5f54ff64f60f None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:52.928313 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f4fef-dae7-4f59-9c05-5f54ff64f60f None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:52.929942 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 709/1410] 10.222.0.33 () {54 vars in 918 bytes} [Thu Jun 2 19:13:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 02 19:13:52.936413 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e82658-e939-42f5-8c91-7c21e0e71781 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:52.937058 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e82658-e939-42f5-8c91-7c21e0e71781 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:52.937058 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e82658-e939-42f5-8c91-7c21e0e71781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:52.956698 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-03e82658-e939-42f5-8c91-7c21e0e71781 None None] MFA Rules not processed for user `73ae837e99764bebbe7a6e06ac9049e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:13:52.957942 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-03e82658-e939-42f5-8c91-7c21e0e71781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:13:52.967400 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 702/1411] 10.222.0.33 () {58 vars in 1015 bytes} [Thu Jun 2 19:13:52 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 19:13:52.983155 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854982b-18dc-4505-8e4b-7632cc0c3a33 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:52.983487 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854982b-18dc-4505-8e4b-7632cc0c3a33 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:52.983806 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854982b-18dc-4505-8e4b-7632cc0c3a33 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:52.985243 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 710/1412] 10.222.0.33 () {54 vars in 918 bytes} [Thu Jun 2 19:13:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 02 19:13:52.991163 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89e0212-e8a2-47c0-908d-fbaf5888d599 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:52.991495 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89e0212-e8a2-47c0-908d-fbaf5888d599 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:52.991817 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89e0212-e8a2-47c0-908d-fbaf5888d599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:53.011254 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-c89e0212-e8a2-47c0-908d-fbaf5888d599 None None] MFA Rules not processed for user `70664d5cbdee47ee89c120761939cc75`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:13:53.013160 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-c89e0212-e8a2-47c0-908d-fbaf5888d599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:13:53.022574 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 703/1413] 10.222.0.33 () {58 vars in 1015 bytes} [Thu Jun 2 19:13:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 19:13:55.843121 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c32039b-833e-4d3e-ace5-6de860026664 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:55.843983 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c32039b-833e-4d3e-ace5-6de860026664 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:55.843983 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c32039b-833e-4d3e-ace5-6de860026664 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:55.845344 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 711/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 2 19:13:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:13:55.852068 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc4a70-abff-4a31-b009-137b52baa71b None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:55.852767 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc4a70-abff-4a31-b009-137b52baa71b None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:55.852767 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc4a70-abff-4a31-b009-137b52baa71b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:55.874303 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-88fc4a70-abff-4a31-b009-137b52baa71b None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:13:55.875599 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-88fc4a70-abff-4a31-b009-137b52baa71b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:13:55.883907 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 704/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 2 19:13:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 19:13:55.931075 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c32039b-833e-4d3e-ace5-6de860026664 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:13:55.954457 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c32039b-833e-4d3e-ace5-6de860026664 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:13:55.960565 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef453c69-5e0d-4366-ac41-22ce8f320abf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:13:55.961249 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef453c69-5e0d-4366-ac41-22ce8f320abf service nova] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:55.961534 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef453c69-5e0d-4366-ac41-22ce8f320abf service nova] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:55.961812 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef453c69-5e0d-4366-ac41-22ce8f320abf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:55.971821 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef453c69-5e0d-4366-ac41-22ce8f320abf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:13:55.976823 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef453c69-5e0d-4366-ac41-22ce8f320abf service nova] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:13:55.985123 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 712/1416] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:13:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:13:57.008299 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da920a6-87e2-4a44-9676-ba9a1283170b None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:57.009050 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da920a6-87e2-4a44-9676-ba9a1283170b None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:57.009190 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da920a6-87e2-4a44-9676-ba9a1283170b None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:57.011497 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 705/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 2 19:13:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:13:57.017990 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4614c9a9-2ab8-402c-b603-0713b0817f3b None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:57.018802 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4614c9a9-2ab8-402c-b603-0713b0817f3b None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:57.018802 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4614c9a9-2ab8-402c-b603-0713b0817f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:57.041888 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-4614c9a9-2ab8-402c-b603-0713b0817f3b None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:13:57.043180 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-4614c9a9-2ab8-402c-b603-0713b0817f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:13:57.052145 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 713/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 2 19:13:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:13:57.221389 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da920a6-87e2-4a44-9676-ba9a1283170b None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:13:57.236689 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da920a6-87e2-4a44-9676-ba9a1283170b None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:13:57.242429 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b826738a-02f6-48e4-ae63-7267bf317873 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:13:57.243121 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b826738a-02f6-48e4-ae63-7267bf317873 service nova] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:57.243407 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b826738a-02f6-48e4-ae63-7267bf317873 service nova] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:57.243685 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b826738a-02f6-48e4-ae63-7267bf317873 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:57.252695 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b826738a-02f6-48e4-ae63-7267bf317873 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:13:57.254297 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b826738a-02f6-48e4-ae63-7267bf317873 service nova] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:13:57.265048 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 706/1419] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:13:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:13:58.450535 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61e1091-0d5d-457e-9b52-7ed481b5f6f1 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:58.451189 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61e1091-0d5d-457e-9b52-7ed481b5f6f1 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:58.451189 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61e1091-0d5d-457e-9b52-7ed481b5f6f1 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:58.452671 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 714/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 2 19:13:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:13:58.458581 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503b2561-ea0d-4e06-a18c-33901f76eb31 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:58.459235 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503b2561-ea0d-4e06-a18c-33901f76eb31 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:58.459235 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503b2561-ea0d-4e06-a18c-33901f76eb31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:58.478435 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-503b2561-ea0d-4e06-a18c-33901f76eb31 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:13:58.479700 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-503b2561-ea0d-4e06-a18c-33901f76eb31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:13:58.488226 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 707/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 2 19:13:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:13:58.645243 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61e1091-0d5d-457e-9b52-7ed481b5f6f1 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:13:58.661134 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61e1091-0d5d-457e-9b52-7ed481b5f6f1 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:13:58.667194 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c6d1e3-6c39-44a5-9cf1-dd447065da78 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:13:58.667927 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6d1e3-6c39-44a5-9cf1-dd447065da78 service nova] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:58.668285 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6d1e3-6c39-44a5-9cf1-dd447065da78 service nova] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:58.669121 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6d1e3-6c39-44a5-9cf1-dd447065da78 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:58.678864 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c6d1e3-6c39-44a5-9cf1-dd447065da78 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:13:58.680570 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c6d1e3-6c39-44a5-9cf1-dd447065da78 service nova] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:13:58.692478 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 715/1422] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:13:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:13:59.648125 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045de931-5f37-4fe7-b39c-ff143653e1af None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:59.648918 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045de931-5f37-4fe7-b39c-ff143653e1af None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:59.648918 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045de931-5f37-4fe7-b39c-ff143653e1af None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:59.650186 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 708/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 2 19:13:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 02 19:13:59.656775 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc5f15-5e1c-463e-8d30-6b78bb094518 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:59.657482 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc5f15-5e1c-463e-8d30-6b78bb094518 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:59.657482 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc5f15-5e1c-463e-8d30-6b78bb094518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:59.676668 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-a1cc5f15-5e1c-463e-8d30-6b78bb094518 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:13:59.677945 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-a1cc5f15-5e1c-463e-8d30-6b78bb094518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:13:59.686315 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 716/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 2 19:13:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 19:13:59.840076 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045de931-5f37-4fe7-b39c-ff143653e1af None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:13:59.854988 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045de931-5f37-4fe7-b39c-ff143653e1af None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:13:59.861269 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12278f2f-b4d5-4795-b901-ce855405aed7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:13:59.861950 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12278f2f-b4d5-4795-b901-ce855405aed7 service nova] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:13:59.862236 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12278f2f-b4d5-4795-b901-ce855405aed7 service nova] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:13:59.862512 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12278f2f-b4d5-4795-b901-ce855405aed7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:13:59.872108 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12278f2f-b4d5-4795-b901-ce855405aed7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:13:59.873907 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12278f2f-b4d5-4795-b901-ce855405aed7 service nova] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:13:59.884378 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 709/1425] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:13:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:14:00.868523 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7ba6f1-e6f4-46f6-beb6-1086405990f7 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:14:00.869065 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7ba6f1-e6f4-46f6-beb6-1086405990f7 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:14:00.869065 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7ba6f1-e6f4-46f6-beb6-1086405990f7 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:14:00.870441 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 717/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 2 19:14:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:14:00.876605 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36af7c8-2ed7-4580-92cf-ab027c75069b None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:14:00.877138 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36af7c8-2ed7-4580-92cf-ab027c75069b None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:14:00.877261 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36af7c8-2ed7-4580-92cf-ab027c75069b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:14:00.895911 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-a36af7c8-2ed7-4580-92cf-ab027c75069b None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:14:00.897141 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-a36af7c8-2ed7-4580-92cf-ab027c75069b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:14:00.905082 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 710/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 2 19:14:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 19:14:01.063599 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7ba6f1-e6f4-46f6-beb6-1086405990f7 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:14:01.080200 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7ba6f1-e6f4-46f6-beb6-1086405990f7 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:14:01.085901 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23019be7-3fea-4754-b7f7-cb17268228a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:14:01.086695 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23019be7-3fea-4754-b7f7-cb17268228a5 service nova] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:14:01.087022 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23019be7-3fea-4754-b7f7-cb17268228a5 service nova] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:14:01.087304 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23019be7-3fea-4754-b7f7-cb17268228a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:14:01.097743 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23019be7-3fea-4754-b7f7-cb17268228a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:14:01.099439 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23019be7-3fea-4754-b7f7-cb17268228a5 service nova] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:14:01.110422 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 718/1428] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:14:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:14:02.054728 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e5248c-bdc8-457a-8c97-f36cbe0529c1 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:14:02.055386 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e5248c-bdc8-457a-8c97-f36cbe0529c1 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:14:02.055386 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e5248c-bdc8-457a-8c97-f36cbe0529c1 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:14:02.056711 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 711/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 2 19:14:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:14:02.063047 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9584a975-b904-4b06-a2e5-2d7c0c03f3e4 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:14:02.063666 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9584a975-b904-4b06-a2e5-2d7c0c03f3e4 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:14:02.063828 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9584a975-b904-4b06-a2e5-2d7c0c03f3e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:14:02.086521 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-9584a975-b904-4b06-a2e5-2d7c0c03f3e4 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:14:02.087789 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-9584a975-b904-4b06-a2e5-2d7c0c03f3e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:14:02.096535 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 719/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 2 19:14:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:14:02.256009 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e5248c-bdc8-457a-8c97-f36cbe0529c1 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:14:02.271814 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e5248c-bdc8-457a-8c97-f36cbe0529c1 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:14:02.278201 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64033158-e21c-4e3c-bc85-d1b2e2763a17 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:14:02.279043 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64033158-e21c-4e3c-bc85-d1b2e2763a17 service nova] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:14:02.279319 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64033158-e21c-4e3c-bc85-d1b2e2763a17 service nova] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:14:02.279597 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64033158-e21c-4e3c-bc85-d1b2e2763a17 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:14:02.288792 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64033158-e21c-4e3c-bc85-d1b2e2763a17 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:14:02.290396 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64033158-e21c-4e3c-bc85-d1b2e2763a17 service nova] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:14:02.300988 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 712/1431] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:14:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:14:03.293384 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7733a27e-e7bb-45e8-a338-6e944fe296d1 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:14:03.293962 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7733a27e-e7bb-45e8-a338-6e944fe296d1 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:14:03.293962 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7733a27e-e7bb-45e8-a338-6e944fe296d1 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:14:03.295376 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 720/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 2 19:14:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:14:03.300358 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2bb70f-b493-41ed-a65d-847f9213fd4c None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:14:03.300816 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2bb70f-b493-41ed-a65d-847f9213fd4c None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:14:03.300942 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2bb70f-b493-41ed-a65d-847f9213fd4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:14:03.318927 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-4d2bb70f-b493-41ed-a65d-847f9213fd4c None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:14:03.320160 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-4d2bb70f-b493-41ed-a65d-847f9213fd4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:14:03.328714 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 713/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 2 19:14:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:14:03.479541 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7733a27e-e7bb-45e8-a338-6e944fe296d1 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:14:03.493980 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7733a27e-e7bb-45e8-a338-6e944fe296d1 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:14:03.499938 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1804dbe2-4dc4-428c-99cf-c6a7966003b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:14:03.500587 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1804dbe2-4dc4-428c-99cf-c6a7966003b2 service nova] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:14:03.500869 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1804dbe2-4dc4-428c-99cf-c6a7966003b2 service nova] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:14:03.501147 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1804dbe2-4dc4-428c-99cf-c6a7966003b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:14:03.509862 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1804dbe2-4dc4-428c-99cf-c6a7966003b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:14:03.511491 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1804dbe2-4dc4-428c-99cf-c6a7966003b2 service nova] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:14:03.521894 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 721/1434] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:14:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:14:04.476606 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce9a50d-a699-4178-a2a1-4e59002df266 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:14:04.477177 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce9a50d-a699-4178-a2a1-4e59002df266 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:14:04.477177 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce9a50d-a699-4178-a2a1-4e59002df266 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:14:04.478551 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 714/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 2 19:14:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:14:04.484280 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1097ffc3-faa1-4a13-91c8-733e262eeb39 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:14:04.484760 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1097ffc3-faa1-4a13-91c8-733e262eeb39 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:14:04.484856 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1097ffc3-faa1-4a13-91c8-733e262eeb39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:14:04.502880 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-1097ffc3-faa1-4a13-91c8-733e262eeb39 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:14:04.504098 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-1097ffc3-faa1-4a13-91c8-733e262eeb39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:14:04.511774 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 722/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 2 19:14:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:14:04.662559 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce9a50d-a699-4178-a2a1-4e59002df266 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:14:04.677859 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce9a50d-a699-4178-a2a1-4e59002df266 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:14:04.684247 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b88b1cc-9e90-47f8-b2e6-8f14a8ea6f75 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:14:04.684919 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b88b1cc-9e90-47f8-b2e6-8f14a8ea6f75 service nova] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:14:04.685207 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b88b1cc-9e90-47f8-b2e6-8f14a8ea6f75 service nova] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:14:04.685491 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b88b1cc-9e90-47f8-b2e6-8f14a8ea6f75 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:14:04.694913 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b88b1cc-9e90-47f8-b2e6-8f14a8ea6f75 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:14:04.696528 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b88b1cc-9e90-47f8-b2e6-8f14a8ea6f75 service nova] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:14:04.707109 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 715/1437] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:14:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:14:05.636482 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aba998-3815-4e99-8c80-72f818128424 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:14:05.637078 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aba998-3815-4e99-8c80-72f818128424 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:14:05.637078 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aba998-3815-4e99-8c80-72f818128424 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:14:05.638530 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 723/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 2 19:14:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:14:05.643882 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfcb2ce-0678-458e-aeb8-c731be48cb00 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:14:05.649700 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfcb2ce-0678-458e-aeb8-c731be48cb00 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:14:05.649700 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfcb2ce-0678-458e-aeb8-c731be48cb00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:14:05.662665 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-9dfcb2ce-0678-458e-aeb8-c731be48cb00 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:14:05.663921 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-9dfcb2ce-0678-458e-aeb8-c731be48cb00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:14:05.672755 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 716/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 2 19:14:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:14:05.848772 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aba998-3815-4e99-8c80-72f818128424 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:14:05.868542 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aba998-3815-4e99-8c80-72f818128424 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:14:05.874026 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06169d8d-3213-40ea-a184-d4b08bafa7b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:14:05.874863 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06169d8d-3213-40ea-a184-d4b08bafa7b3 service nova] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:14:05.875002 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06169d8d-3213-40ea-a184-d4b08bafa7b3 service nova] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:14:05.875267 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06169d8d-3213-40ea-a184-d4b08bafa7b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:14:05.884256 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06169d8d-3213-40ea-a184-d4b08bafa7b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:14:05.885941 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06169d8d-3213-40ea-a184-d4b08bafa7b3 service nova] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:14:05.897513 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 724/1440] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:14:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:14:08.087794 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eec6c7-ab1f-4e85-b626-b80baa1af02c None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:14:08.088369 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eec6c7-ab1f-4e85-b626-b80baa1af02c None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:14:08.088369 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eec6c7-ab1f-4e85-b626-b80baa1af02c None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:14:08.089757 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 717/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 2 19:14:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:14:08.095318 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9dd5f-b807-4870-b1a7-211a78b95345 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:14:08.095734 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9dd5f-b807-4870-b1a7-211a78b95345 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:14:08.095890 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9dd5f-b807-4870-b1a7-211a78b95345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:14:08.114275 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-cdd9dd5f-b807-4870-b1a7-211a78b95345 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:14:08.115581 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-cdd9dd5f-b807-4870-b1a7-211a78b95345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:14:08.124246 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 725/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 2 19:14:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:14:08.162727 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eec6c7-ab1f-4e85-b626-b80baa1af02c None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:14:08.173376 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eec6c7-ab1f-4e85-b626-b80baa1af02c None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:14:08.176835 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69cf538-6752-4e3c-add9-f80cf7d9e395 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:14:08.177468 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69cf538-6752-4e3c-add9-f80cf7d9e395 service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:14:08.177749 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69cf538-6752-4e3c-add9-f80cf7d9e395 service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:14:08.178020 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69cf538-6752-4e3c-add9-f80cf7d9e395 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:14:08.187438 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69cf538-6752-4e3c-add9-f80cf7d9e395 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:14:08.188852 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69cf538-6752-4e3c-add9-f80cf7d9e395 service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:14:08.198112 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 718/1443] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:14:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:14:10.630064 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d2f44-85ed-4b29-bf81-088327d27e99 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:14:10.630648 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d2f44-85ed-4b29-bf81-088327d27e99 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:14:10.630648 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d2f44-85ed-4b29-bf81-088327d27e99 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:14:10.632040 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 726/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 2 19:14:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:14:10.638314 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7d6fc-3168-4484-8be2-8add767824e8 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:14:10.638853 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7d6fc-3168-4484-8be2-8add767824e8 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:14:10.638960 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7d6fc-3168-4484-8be2-8add767824e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:14:10.658659 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-d9d7d6fc-3168-4484-8be2-8add767824e8 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:14:10.659922 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-d9d7d6fc-3168-4484-8be2-8add767824e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:14:10.667852 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 719/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 2 19:14:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:14:10.706990 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44d2f44-85ed-4b29-bf81-088327d27e99 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:14:10.718426 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44d2f44-85ed-4b29-bf81-088327d27e99 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:14:10.721947 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa08d16f-d804-47b6-b788-a81d1ff8beea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:14:10.722590 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa08d16f-d804-47b6-b788-a81d1ff8beea service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:14:10.722903 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa08d16f-d804-47b6-b788-a81d1ff8beea service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:14:10.723182 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa08d16f-d804-47b6-b788-a81d1ff8beea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:14:10.732858 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa08d16f-d804-47b6-b788-a81d1ff8beea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:14:10.734312 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa08d16f-d804-47b6-b788-a81d1ff8beea service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:14:10.744551 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 727/1446] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:14:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:14:13.418979 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ff53f-9cde-4534-99a1-120003acef76 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:14:13.419613 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ff53f-9cde-4534-99a1-120003acef76 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:14:13.419613 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ff53f-9cde-4534-99a1-120003acef76 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:14:13.420858 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 720/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 2 19:14:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:14:13.426951 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd17a2fd-9e96-4f5c-972c-dff0b08c05a6 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:14:13.427554 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd17a2fd-9e96-4f5c-972c-dff0b08c05a6 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:14:13.427554 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd17a2fd-9e96-4f5c-972c-dff0b08c05a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:14:13.446616 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-bd17a2fd-9e96-4f5c-972c-dff0b08c05a6 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:14:13.447912 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-bd17a2fd-9e96-4f5c-972c-dff0b08c05a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:14:13.457363 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 728/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 2 19:14:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:14:13.497804 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9ff53f-9cde-4534-99a1-120003acef76 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:14:13.508788 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9ff53f-9cde-4534-99a1-120003acef76 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:14:13.513033 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4acbb32-0b8a-44d1-bdbe-ecb8b4566b0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:14:13.513691 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4acbb32-0b8a-44d1-bdbe-ecb8b4566b0f service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:14:13.513978 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4acbb32-0b8a-44d1-bdbe-ecb8b4566b0f service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:14:13.514259 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4acbb32-0b8a-44d1-bdbe-ecb8b4566b0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:14:13.523996 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4acbb32-0b8a-44d1-bdbe-ecb8b4566b0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:14:13.525435 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4acbb32-0b8a-44d1-bdbe-ecb8b4566b0f service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:14:13.536073 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 721/1449] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:14:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:14:15.980044 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa2320-40a8-411f-9a87-e43255960e9c None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:14:15.981458 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa2320-40a8-411f-9a87-e43255960e9c None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:14:15.981640 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa2320-40a8-411f-9a87-e43255960e9c None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:14:15.983426 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 729/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 2 19:14:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:14:15.989516 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9b0152-ca29-4e35-b760-73b21022f7cf None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:14:15.990074 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9b0152-ca29-4e35-b760-73b21022f7cf None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:14:15.990074 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9b0152-ca29-4e35-b760-73b21022f7cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:14:16.008540 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-9f9b0152-ca29-4e35-b760-73b21022f7cf None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:14:16.009761 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-9f9b0152-ca29-4e35-b760-73b21022f7cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:14:16.018385 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 722/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 2 19:14:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:14:16.057666 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aa2320-40a8-411f-9a87-e43255960e9c None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:14:16.059486 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-29aa2320-40a8-411f-9a87-e43255960e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:14:16.072480 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aa2320-40a8-411f-9a87-e43255960e9c None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:14:16.076752 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7feb76-2b67-430f-8672-c76796335e2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:14:16.077402 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7feb76-2b67-430f-8672-c76796335e2b service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:14:16.077710 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7feb76-2b67-430f-8672-c76796335e2b service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:14:16.077982 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7feb76-2b67-430f-8672-c76796335e2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:14:16.087729 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7feb76-2b67-430f-8672-c76796335e2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:14:16.089527 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7feb76-2b67-430f-8672-c76796335e2b service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:14:16.099972 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 730/1452] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:14:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:14:18.596154 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff4dc7f-5f9c-4216-a961-5bf74ded640d None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:14:18.596694 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff4dc7f-5f9c-4216-a961-5bf74ded640d None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:14:18.596694 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff4dc7f-5f9c-4216-a961-5bf74ded640d None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:14:18.598125 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 723/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 2 19:14:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:14:18.603885 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbea44e-f59b-4bf6-9787-a54bf37d5a31 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:14:18.604396 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbea44e-f59b-4bf6-9787-a54bf37d5a31 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:14:18.604504 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbea44e-f59b-4bf6-9787-a54bf37d5a31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:14:18.624501 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-1bbea44e-f59b-4bf6-9787-a54bf37d5a31 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:14:18.625736 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-1bbea44e-f59b-4bf6-9787-a54bf37d5a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:14:18.633901 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 731/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 2 19:14:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 19:14:18.672456 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff4dc7f-5f9c-4216-a961-5bf74ded640d None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:14:18.683746 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff4dc7f-5f9c-4216-a961-5bf74ded640d None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:14:18.687722 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b449b4-c1e3-4cb5-846c-804194f1af11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:14:18.688370 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b449b4-c1e3-4cb5-846c-804194f1af11 service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:14:18.688645 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b449b4-c1e3-4cb5-846c-804194f1af11 service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:14:18.688917 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b449b4-c1e3-4cb5-846c-804194f1af11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:14:18.701941 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b449b4-c1e3-4cb5-846c-804194f1af11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:14:18.703975 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b449b4-c1e3-4cb5-846c-804194f1af11 service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:14:18.714995 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 724/1455] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:14:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:14:21.156166 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8104011-feb0-45b1-8b45-fe56c026c624 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:14:21.156737 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8104011-feb0-45b1-8b45-fe56c026c624 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:14:21.156737 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8104011-feb0-45b1-8b45-fe56c026c624 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:14:21.158111 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 732/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 2 19:14:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:14:21.163673 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749bf083-0551-4e5f-ac9a-8d396cc298cc None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:14:21.164310 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749bf083-0551-4e5f-ac9a-8d396cc298cc None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:14:21.164310 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749bf083-0551-4e5f-ac9a-8d396cc298cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:14:21.181938 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-749bf083-0551-4e5f-ac9a-8d396cc298cc None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:14:21.183173 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-749bf083-0551-4e5f-ac9a-8d396cc298cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:14:21.191420 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 725/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 2 19:14:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:14:21.230401 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8104011-feb0-45b1-8b45-fe56c026c624 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:14:21.241684 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8104011-feb0-45b1-8b45-fe56c026c624 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:14:21.245312 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9d6f2d-34fd-4caf-9965-a437a2bf779e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:14:21.245970 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9d6f2d-34fd-4caf-9965-a437a2bf779e service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:14:21.246256 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9d6f2d-34fd-4caf-9965-a437a2bf779e service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:14:21.246551 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9d6f2d-34fd-4caf-9965-a437a2bf779e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:14:21.257528 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9d6f2d-34fd-4caf-9965-a437a2bf779e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:14:21.259029 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9d6f2d-34fd-4caf-9965-a437a2bf779e service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:14:21.270171 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 733/1458] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:14:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:14:23.629244 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1babe8ef-f41a-4616-8018-fe2d006b358d None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:14:23.629829 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1babe8ef-f41a-4616-8018-fe2d006b358d None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:14:23.629829 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1babe8ef-f41a-4616-8018-fe2d006b358d None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:14:23.631382 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 726/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 2 19:14:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:14:23.637821 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17cd28c-fbe8-4dd0-85a8-b8d4ad152387 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:14:23.638328 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17cd28c-fbe8-4dd0-85a8-b8d4ad152387 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:14:23.638443 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17cd28c-fbe8-4dd0-85a8-b8d4ad152387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:14:23.644643 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=70 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:14:23.657433 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-c17cd28c-fbe8-4dd0-85a8-b8d4ad152387 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:14:23.666235 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-c17cd28c-fbe8-4dd0-85a8-b8d4ad152387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:14:23.667405 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 734/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 2 19:14:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:14:23.705244 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1babe8ef-f41a-4616-8018-fe2d006b358d None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:14:23.710626 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=73 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:14:23.717159 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1babe8ef-f41a-4616-8018-fe2d006b358d None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:14:23.724518 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc5b75e-913f-4a09-8ff7-aa0de0f6397c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:14:23.724518 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc5b75e-913f-4a09-8ff7-aa0de0f6397c service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:14:23.724518 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc5b75e-913f-4a09-8ff7-aa0de0f6397c service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:14:23.724518 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc5b75e-913f-4a09-8ff7-aa0de0f6397c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:14:23.731925 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc5b75e-913f-4a09-8ff7-aa0de0f6397c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:14:23.733380 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc5b75e-913f-4a09-8ff7-aa0de0f6397c service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:14:23.746130 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 727/1461] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:14:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 02 19:14:26.063364 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b38e9b-c03d-4516-8084-dfeb858668c3 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:14:26.063986 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b38e9b-c03d-4516-8084-dfeb858668c3 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:14:26.063986 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b38e9b-c03d-4516-8084-dfeb858668c3 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:14:26.065588 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 735/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 2 19:14:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:14:26.071557 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fbdc89-7dd0-4d77-b054-cddeadfd7149 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:14:26.072077 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fbdc89-7dd0-4d77-b054-cddeadfd7149 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:14:26.072190 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fbdc89-7dd0-4d77-b054-cddeadfd7149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:14:26.090056 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-a9fbdc89-7dd0-4d77-b054-cddeadfd7149 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:14:26.091818 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-a9fbdc89-7dd0-4d77-b054-cddeadfd7149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:14:26.099774 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 728/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 2 19:14:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:14:26.137840 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b38e9b-c03d-4516-8084-dfeb858668c3 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:14:26.148693 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b38e9b-c03d-4516-8084-dfeb858668c3 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:14:26.152581 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ccf2e6-57f5-4598-92e0-b87c90135177 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:14:26.153757 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ccf2e6-57f5-4598-92e0-b87c90135177 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:14:26.154043 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ccf2e6-57f5-4598-92e0-b87c90135177 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:14:26.154320 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ccf2e6-57f5-4598-92e0-b87c90135177 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:14:26.164065 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ccf2e6-57f5-4598-92e0-b87c90135177 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:14:26.165535 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ccf2e6-57f5-4598-92e0-b87c90135177 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:14:26.176153 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 736/1464] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:14:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:14:28.512767 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad7d96-5c81-4e61-a0db-1f2f31aff237 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:14:28.513329 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad7d96-5c81-4e61-a0db-1f2f31aff237 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:14:28.513329 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad7d96-5c81-4e61-a0db-1f2f31aff237 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:14:28.514739 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 729/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 2 19:14:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:14:28.520469 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b473d3-d66d-4870-9038-454f4589af71 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:14:28.521012 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b473d3-d66d-4870-9038-454f4589af71 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:14:28.521012 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b473d3-d66d-4870-9038-454f4589af71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:14:28.540347 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-54b473d3-d66d-4870-9038-454f4589af71 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:14:28.541595 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-54b473d3-d66d-4870-9038-454f4589af71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:14:28.550306 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 737/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 2 19:14:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:14:28.587395 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ad7d96-5c81-4e61-a0db-1f2f31aff237 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:14:28.598913 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ad7d96-5c81-4e61-a0db-1f2f31aff237 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:14:28.602423 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd73738c-446d-416c-a8a0-7ec95841b426 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:14:28.603106 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd73738c-446d-416c-a8a0-7ec95841b426 service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:14:28.603393 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd73738c-446d-416c-a8a0-7ec95841b426 service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:14:28.603672 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd73738c-446d-416c-a8a0-7ec95841b426 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:14:28.614048 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd73738c-446d-416c-a8a0-7ec95841b426 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:14:28.615529 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd73738c-446d-416c-a8a0-7ec95841b426 service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:14:28.625489 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 730/1467] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:14:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:14:30.911403 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e1aa12-3f1d-48b1-8588-4cd0f6213afe None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:14:30.911953 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e1aa12-3f1d-48b1-8588-4cd0f6213afe None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:14:30.911953 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e1aa12-3f1d-48b1-8588-4cd0f6213afe None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:14:30.914251 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 738/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 2 19:14:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:14:30.920620 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c76f714-4d2f-44c4-a7ef-9cbbe2c1030d None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:14:30.921160 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c76f714-4d2f-44c4-a7ef-9cbbe2c1030d None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:14:30.921268 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c76f714-4d2f-44c4-a7ef-9cbbe2c1030d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:14:30.940259 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-7c76f714-4d2f-44c4-a7ef-9cbbe2c1030d None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:14:30.941591 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-7c76f714-4d2f-44c4-a7ef-9cbbe2c1030d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:14:30.950038 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 731/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 2 19:14:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:14:30.987381 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e1aa12-3f1d-48b1-8588-4cd0f6213afe None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:14:30.999936 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e1aa12-3f1d-48b1-8588-4cd0f6213afe None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:14:31.004067 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a14cac3-c9d4-4168-901d-e12db412538c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:14:31.004739 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a14cac3-c9d4-4168-901d-e12db412538c service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:14:31.005027 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a14cac3-c9d4-4168-901d-e12db412538c service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:14:31.005307 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a14cac3-c9d4-4168-901d-e12db412538c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:14:31.015561 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a14cac3-c9d4-4168-901d-e12db412538c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:14:31.017044 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a14cac3-c9d4-4168-901d-e12db412538c service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:14:31.027083 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 739/1470] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:14:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:14:40.933604 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=17 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:14:41.022552 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=17 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:15:03.590728 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121fbaa9-7040-44d5-bcd0-252d04f6426c None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:15:03.591374 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121fbaa9-7040-44d5-bcd0-252d04f6426c None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:15:03.591374 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121fbaa9-7040-44d5-bcd0-252d04f6426c None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:15:03.592751 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 732/1471] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:15:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 02 19:15:03.599218 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381144c3-475a-4f62-ac87-7db4c8bd7705 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:15:03.599706 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381144c3-475a-4f62-ac87-7db4c8bd7705 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:15:03.599818 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381144c3-475a-4f62-ac87-7db4c8bd7705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:15:03.618294 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-381144c3-475a-4f62-ac87-7db4c8bd7705 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:15:03.619533 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-381144c3-475a-4f62-ac87-7db4c8bd7705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:15:03.628904 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 740/1472] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:15:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:15:03.678512 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99346445-d630-4c25-94f5-e24e360c9de9 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:15:03.678902 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99346445-d630-4c25-94f5-e24e360c9de9 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:15:03.679226 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99346445-d630-4c25-94f5-e24e360c9de9 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:15:03.680749 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 733/1473] 10.222.0.33 () {54 vars in 962 bytes} [Thu Jun 2 19:15:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:15:03.686234 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27606e5b-c4fd-461d-a888-df626d618cbb None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:15:03.686575 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27606e5b-c4fd-461d-a888-df626d618cbb None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:15:03.686855 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27606e5b-c4fd-461d-a888-df626d618cbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:15:03.705868 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-27606e5b-c4fd-461d-a888-df626d618cbb None None] MFA Rules not processed for user `e07892c0aa21449b94a1cc78e3b8a653`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:15:03.707152 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-27606e5b-c4fd-461d-a888-df626d618cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:15:03.713847 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 741/1474] 10.222.0.33 () {58 vars in 1059 bytes} [Thu Jun 2 19:15:03 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:15:03.722054 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99346445-d630-4c25-94f5-e24e360c9de9 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:15:03.735518 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99346445-d630-4c25-94f5-e24e360c9de9 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:15:03.739478 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b119114-11de-4a0f-8265-d780bb328f54 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07892c0aa21449b94a1cc78e3b8a653', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:15:03.740138 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b119114-11de-4a0f-8265-d780bb328f54 service glance] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:15:03.740428 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b119114-11de-4a0f-8265-d780bb328f54 service glance] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:15:03.740712 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b119114-11de-4a0f-8265-d780bb328f54 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:15:03.752033 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b119114-11de-4a0f-8265-d780bb328f54 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:15:03.752464 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b119114-11de-4a0f-8265-d780bb328f54 service glance] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:15:03.763407 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 734/1475] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:15:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:15:06.197953 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28df686-6e0b-4dff-9e1a-3739a674503b None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:15:06.198515 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28df686-6e0b-4dff-9e1a-3739a674503b None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:15:06.198515 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28df686-6e0b-4dff-9e1a-3739a674503b None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:15:06.200583 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 742/1476] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:15:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:15:06.206633 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b5033e-92c2-42c7-a1e3-2a5b27fd82fd None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:15:06.207287 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b5033e-92c2-42c7-a1e3-2a5b27fd82fd None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:15:06.207287 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b5033e-92c2-42c7-a1e3-2a5b27fd82fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:15:06.230591 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-e4b5033e-92c2-42c7-a1e3-2a5b27fd82fd None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:15:06.231915 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-e4b5033e-92c2-42c7-a1e3-2a5b27fd82fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:15:06.240636 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 735/1477] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:15:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:15:06.266048 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28df686-6e0b-4dff-9e1a-3739a674503b None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:15:06.279652 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28df686-6e0b-4dff-9e1a-3739a674503b None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:15:06.284391 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219b8ae0-86d4-40a7-9bc1-51a3437866da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:15:06.285061 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219b8ae0-86d4-40a7-9bc1-51a3437866da admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:15:06.285350 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219b8ae0-86d4-40a7-9bc1-51a3437866da admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:15:06.285629 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219b8ae0-86d4-40a7-9bc1-51a3437866da admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:15:06.286380 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219b8ae0-86d4-40a7-9bc1-51a3437866da admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:15:06.288471 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: WARNING py.warnings [None req-219b8ae0-86d4-40a7-9bc1-51a3437866da admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 02 19:15:06.288471 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: warnings.warn(msg) Jun 02 19:15:06.288471 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: Jun 02 19:15:06.289624 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219b8ae0-86d4-40a7-9bc1-51a3437866da admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:15:06.310014 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 743/1478] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Jun 2 19:15:06 2022] GET /identity/v3/projects => generated 2553 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:15:08.671724 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505ebde6-ca24-4f69-9ad9-bfeb6e615327 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:15:08.672288 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505ebde6-ca24-4f69-9ad9-bfeb6e615327 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:15:08.672288 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505ebde6-ca24-4f69-9ad9-bfeb6e615327 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:15:08.673754 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 736/1479] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:15:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:15:08.679909 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c2c4e-87be-412e-a13f-4fb0150e3e00 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:15:08.680474 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c2c4e-87be-412e-a13f-4fb0150e3e00 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:15:08.680474 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c2c4e-87be-412e-a13f-4fb0150e3e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:15:08.700053 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-9f7c2c4e-87be-412e-a13f-4fb0150e3e00 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:15:08.701304 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-9f7c2c4e-87be-412e-a13f-4fb0150e3e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:15:08.709614 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 744/1480] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:15:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 19:15:08.755868 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505ebde6-ca24-4f69-9ad9-bfeb6e615327 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:15:08.770519 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505ebde6-ca24-4f69-9ad9-bfeb6e615327 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:15:08.775992 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078ce061-cbff-496c-add0-9d5f0345b1ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:15:08.776661 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078ce061-cbff-496c-add0-9d5f0345b1ab service nova] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:15:08.776946 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078ce061-cbff-496c-add0-9d5f0345b1ab service nova] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:15:08.777378 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078ce061-cbff-496c-add0-9d5f0345b1ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:15:08.786983 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078ce061-cbff-496c-add0-9d5f0345b1ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:15:08.788609 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078ce061-cbff-496c-add0-9d5f0345b1ab service nova] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:15:08.799888 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 737/1481] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:15:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 02 19:15:11.297176 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c0f37e-f61b-4d7b-9e29-c7da44c4935c None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:15:11.297767 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c0f37e-f61b-4d7b-9e29-c7da44c4935c None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:15:11.297767 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c0f37e-f61b-4d7b-9e29-c7da44c4935c None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:15:11.299152 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 745/1482] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:15:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:15:11.304864 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42dcf95-9bd9-4969-bc1c-c9ec356a24e9 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:15:11.305365 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42dcf95-9bd9-4969-bc1c-c9ec356a24e9 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:15:11.305506 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42dcf95-9bd9-4969-bc1c-c9ec356a24e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:15:11.323941 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-c42dcf95-9bd9-4969-bc1c-c9ec356a24e9 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:15:11.325199 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-c42dcf95-9bd9-4969-bc1c-c9ec356a24e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:15:11.333669 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 738/1483] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:15:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:15:11.371339 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c0f37e-f61b-4d7b-9e29-c7da44c4935c None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:15:11.382181 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c0f37e-f61b-4d7b-9e29-c7da44c4935c None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:15:11.385713 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2b7bf0-6b4e-455f-a58e-4c863e44a452 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:15:11.388042 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2b7bf0-6b4e-455f-a58e-4c863e44a452 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:15:11.388042 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2b7bf0-6b4e-455f-a58e-4c863e44a452 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:15:11.388042 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2b7bf0-6b4e-455f-a58e-4c863e44a452 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:15:11.397141 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2b7bf0-6b4e-455f-a58e-4c863e44a452 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:15:11.398649 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2b7bf0-6b4e-455f-a58e-4c863e44a452 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:15:11.409522 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 746/1484] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:15:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:15:13.793282 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f6434c-a9d9-43ad-8b82-90b2cb817ed1 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:15:13.794012 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f6434c-a9d9-43ad-8b82-90b2cb817ed1 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:15:13.794120 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f6434c-a9d9-43ad-8b82-90b2cb817ed1 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:15:13.795621 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 739/1485] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:15:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:15:13.801618 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804642c4-0d22-468c-963c-39336e246dbc None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:15:13.802172 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804642c4-0d22-468c-963c-39336e246dbc None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:15:13.802277 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804642c4-0d22-468c-963c-39336e246dbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:15:13.820258 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-804642c4-0d22-468c-963c-39336e246dbc None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:15:13.822594 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-804642c4-0d22-468c-963c-39336e246dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:15:13.830338 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 747/1486] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:15:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:15:13.872125 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f6434c-a9d9-43ad-8b82-90b2cb817ed1 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:15:13.884626 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f6434c-a9d9-43ad-8b82-90b2cb817ed1 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:15:13.889691 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543d8e10-436d-4e66-9a3b-2d8c66d3806a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:15:13.890354 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543d8e10-436d-4e66-9a3b-2d8c66d3806a service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:15:13.890644 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543d8e10-436d-4e66-9a3b-2d8c66d3806a service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:15:13.891028 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543d8e10-436d-4e66-9a3b-2d8c66d3806a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:15:13.904132 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543d8e10-436d-4e66-9a3b-2d8c66d3806a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:15:13.905879 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543d8e10-436d-4e66-9a3b-2d8c66d3806a service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:15:13.917188 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 740/1487] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:15:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:15:16.454980 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02926ae0-b013-46c4-924a-2f0b83ee371a None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:15:16.455533 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02926ae0-b013-46c4-924a-2f0b83ee371a None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:15:16.455533 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02926ae0-b013-46c4-924a-2f0b83ee371a None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:15:16.456852 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 748/1488] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:15:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:15:16.462427 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133fc48f-154e-4d8e-b7ff-52a86b941b2a None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:15:16.463159 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133fc48f-154e-4d8e-b7ff-52a86b941b2a None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:15:16.463159 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133fc48f-154e-4d8e-b7ff-52a86b941b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:15:16.481846 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-133fc48f-154e-4d8e-b7ff-52a86b941b2a None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:15:16.483228 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-133fc48f-154e-4d8e-b7ff-52a86b941b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:15:16.491644 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 741/1489] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:15:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:15:16.533908 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02926ae0-b013-46c4-924a-2f0b83ee371a None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:15:16.546507 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02926ae0-b013-46c4-924a-2f0b83ee371a None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:15:16.550618 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca4986e-c877-4439-9c8e-38da2bfbf240 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:15:16.551316 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4986e-c877-4439-9c8e-38da2bfbf240 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:15:16.551603 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4986e-c877-4439-9c8e-38da2bfbf240 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:15:16.551882 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4986e-c877-4439-9c8e-38da2bfbf240 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:15:16.562254 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca4986e-c877-4439-9c8e-38da2bfbf240 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:15:16.563792 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca4986e-c877-4439-9c8e-38da2bfbf240 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:15:16.574443 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 749/1490] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:15:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:15:19.356191 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e52dc6-bf89-4bc4-84d3-6a657da24155 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:15:19.356783 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e52dc6-bf89-4bc4-84d3-6a657da24155 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:15:19.356783 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e52dc6-bf89-4bc4-84d3-6a657da24155 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:15:19.358129 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 742/1491] 10.222.0.33 () {54 vars in 926 bytes} [Thu Jun 2 19:15:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:15:19.364466 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f22e9f-bca1-4436-8067-690502c033d1 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:15:19.364977 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f22e9f-bca1-4436-8067-690502c033d1 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:15:19.365159 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f22e9f-bca1-4436-8067-690502c033d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:15:19.387065 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-e4f22e9f-bca1-4436-8067-690502c033d1 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:15:19.389124 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-e4f22e9f-bca1-4436-8067-690502c033d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:15:19.398904 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 750/1492] 10.222.0.33 () {58 vars in 1023 bytes} [Thu Jun 2 19:15:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:15:19.440246 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e52dc6-bf89-4bc4-84d3-6a657da24155 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:15:19.452434 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e52dc6-bf89-4bc4-84d3-6a657da24155 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:15:19.456051 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f0beb3-dc86-4764-bd32-765ab6550124 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:15:19.456707 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f0beb3-dc86-4764-bd32-765ab6550124 service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:15:19.456990 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f0beb3-dc86-4764-bd32-765ab6550124 service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:15:19.457268 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f0beb3-dc86-4764-bd32-765ab6550124 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:15:19.467123 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f0beb3-dc86-4764-bd32-765ab6550124 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:15:19.468583 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f0beb3-dc86-4764-bd32-765ab6550124 service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:15:19.478857 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 743/1493] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:15:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:15:29.380883 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=33 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:15:29.474180 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=33 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:16:56.304576 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd48fa-e760-4cd4-bbaa-3e65dfc05f1a None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:16:56.305297 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd48fa-e760-4cd4-bbaa-3e65dfc05f1a None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:16:56.305297 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd48fa-e760-4cd4-bbaa-3e65dfc05f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:16:56.326921 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-cfbd48fa-e760-4cd4-bbaa-3e65dfc05f1a None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:16:56.328127 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-cfbd48fa-e760-4cd4-bbaa-3e65dfc05f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:16:56.335907 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 751/1494] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:16:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:16:56.344564 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ddf237-1a4b-46a8-a806-538fd5a48a13 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:16:56.345257 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ddf237-1a4b-46a8-a806-538fd5a48a13 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:16:56.345257 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ddf237-1a4b-46a8-a806-538fd5a48a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:16:56.364608 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-93ddf237-1a4b-46a8-a806-538fd5a48a13 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:16:56.365819 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-93ddf237-1a4b-46a8-a806-538fd5a48a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:16:56.373984 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 744/1495] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:16:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:16:56.378614 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbd48fa-e760-4cd4-bbaa-3e65dfc05f1a None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:16:56.389201 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbd48fa-e760-4cd4-bbaa-3e65dfc05f1a None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:16:56.393658 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe43bf8-ac9c-46d4-8fcc-dc9de6509c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:16:56.394314 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe43bf8-ac9c-46d4-8fcc-dc9de6509c89 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:16:56.394597 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe43bf8-ac9c-46d4-8fcc-dc9de6509c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:16:56.394936 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe43bf8-ac9c-46d4-8fcc-dc9de6509c89 admin admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:16:56.395660 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe43bf8-ac9c-46d4-8fcc-dc9de6509c89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:16:56.396021 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe43bf8-ac9c-46d4-8fcc-dc9de6509c89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:16:56.396985 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: WARNING py.warnings [None req-4fe43bf8-ac9c-46d4-8fcc-dc9de6509c89 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 02 19:16:56.396985 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: warnings.warn(msg) Jun 02 19:16:56.396985 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: Jun 02 19:16:56.397860 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe43bf8-ac9c-46d4-8fcc-dc9de6509c89 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:16:56.410592 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 752/1496] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 2 19:16:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:16:56.419093 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ddf237-1a4b-46a8-a806-538fd5a48a13 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:16:56.428361 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ddf237-1a4b-46a8-a806-538fd5a48a13 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:16:56.433013 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3eefcd-0f2e-49c7-80e1-65cd68d7e431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:16:56.433671 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3eefcd-0f2e-49c7-80e1-65cd68d7e431 admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:16:56.433984 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3eefcd-0f2e-49c7-80e1-65cd68d7e431 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:16:56.434262 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3eefcd-0f2e-49c7-80e1-65cd68d7e431 admin admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:16:56.435006 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3eefcd-0f2e-49c7-80e1-65cd68d7e431 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:16:56.436757 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: WARNING py.warnings [None req-4b3eefcd-0f2e-49c7-80e1-65cd68d7e431 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 02 19:16:56.436757 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: warnings.warn(msg) Jun 02 19:16:56.436757 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: Jun 02 19:16:56.437939 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3eefcd-0f2e-49c7-80e1-65cd68d7e431 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:16:56.593778 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 745/1497] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:16:56 2022] POST /identity/v3/projects => generated 379 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:16:56.597860 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe43bf8-ac9c-46d4-8fcc-dc9de6509c89 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:16:56.607765 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe43bf8-ac9c-46d4-8fcc-dc9de6509c89 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:16:56.663242 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37320bdb-5053-4181-adf5-a94a8ef2aa85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:16:56.663834 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37320bdb-5053-4181-adf5-a94a8ef2aa85 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:16:56.664123 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37320bdb-5053-4181-adf5-a94a8ef2aa85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:16:56.664400 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37320bdb-5053-4181-adf5-a94a8ef2aa85 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:16:56.665091 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37320bdb-5053-4181-adf5-a94a8ef2aa85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:16:56.666118 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: WARNING py.warnings [None req-37320bdb-5053-4181-adf5-a94a8ef2aa85 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 02 19:16:56.666118 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: warnings.warn(msg) Jun 02 19:16:56.666118 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: Jun 02 19:16:56.667360 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37320bdb-5053-4181-adf5-a94a8ef2aa85 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:16:56.774561 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 753/1498] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:16:56 2022] POST /identity/v3/users => generated 290 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:16:56.779450 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3eefcd-0f2e-49c7-80e1-65cd68d7e431 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:16:56.790061 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3eefcd-0f2e-49c7-80e1-65cd68d7e431 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:16:56.795356 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07490885-7305-4d30-a244-b0c8e2bcb945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:16:56.796026 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07490885-7305-4d30-a244-b0c8e2bcb945 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:16:56.796311 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07490885-7305-4d30-a244-b0c8e2bcb945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:16:56.796594 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07490885-7305-4d30-a244-b0c8e2bcb945 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:16:56.797329 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07490885-7305-4d30-a244-b0c8e2bcb945 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:16:56.798341 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: WARNING py.warnings [None req-07490885-7305-4d30-a244-b0c8e2bcb945 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 02 19:16:56.798341 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: warnings.warn(msg) Jun 02 19:16:56.798341 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: Jun 02 19:16:56.799277 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07490885-7305-4d30-a244-b0c8e2bcb945 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:16:56.812786 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 746/1499] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:16:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:16:56.816981 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37320bdb-5053-4181-adf5-a94a8ef2aa85 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:16:56.827846 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37320bdb-5053-4181-adf5-a94a8ef2aa85 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:16:56.833001 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4701a8-a3b5-4e12-b2a7-14db2bd4b6a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:16:56.833671 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4701a8-a3b5-4e12-b2a7-14db2bd4b6a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:16:56.833955 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4701a8-a3b5-4e12-b2a7-14db2bd4b6a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:16:56.834231 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4701a8-a3b5-4e12-b2a7-14db2bd4b6a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:16:56.834994 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4701a8-a3b5-4e12-b2a7-14db2bd4b6a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:16:56.836014 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: WARNING py.warnings [None req-0f4701a8-a3b5-4e12-b2a7-14db2bd4b6a0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 02 19:16:56.836014 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: warnings.warn(msg) Jun 02 19:16:56.836014 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: Jun 02 19:16:56.836889 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4701a8-a3b5-4e12-b2a7-14db2bd4b6a0 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:16:56.850541 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 754/1500] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:16:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:16:56.854904 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07490885-7305-4d30-a244-b0c8e2bcb945 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:16:56.865179 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07490885-7305-4d30-a244-b0c8e2bcb945 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:16:56.869741 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338fa398-4770-4321-b3c0-96ca81fe947f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:16:56.870625 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338fa398-4770-4321-b3c0-96ca81fe947f admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:16:56.870970 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338fa398-4770-4321-b3c0-96ca81fe947f admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:16:56.871221 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338fa398-4770-4321-b3c0-96ca81fe947f admin admin] PATH_INFO: `/v3/projects/6f74fd99317d48cb86cbb51c6b33fe8d/users/b5ce32a50e9d430c955c5188102e1454/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:16:56.886318 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338fa398-4770-4321-b3c0-96ca81fe947f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f74fd99317d48cb86cbb51c6b33fe8d, user_id=b5ce32a50e9d430c955c5188102e1454, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:16:56.888096 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: WARNING py.warnings [None req-338fa398-4770-4321-b3c0-96ca81fe947f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 02 19:16:56.888096 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: warnings.warn(msg) Jun 02 19:16:56.888096 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: Jun 02 19:16:56.889904 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338fa398-4770-4321-b3c0-96ca81fe947f admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:16:56.974446 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 747/1501] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:16:56 2022] PUT /identity/v3/projects/6f74fd99317d48cb86cbb51c6b33fe8d/users/b5ce32a50e9d430c955c5188102e1454/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:16:56.987023 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b38662-aa65-4b1d-9dc9-762e4788282b None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:16:56.987633 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b38662-aa65-4b1d-9dc9-762e4788282b None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:16:56.987918 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b38662-aa65-4b1d-9dc9-762e4788282b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:16:57.016852 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-a6b38662-aa65-4b1d-9dc9-762e4788282b None None] MFA Rules not processed for user `b5ce32a50e9d430c955c5188102e1454`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:16:57.018128 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-a6b38662-aa65-4b1d-9dc9-762e4788282b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:16:57.083144 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 755/1502] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:16:56 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:16:57.096624 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402b880d-4ed8-4b73-8f7b-5c539264cfdb None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:16:57.096943 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402b880d-4ed8-4b73-8f7b-5c539264cfdb None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:16:57.097224 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402b880d-4ed8-4b73-8f7b-5c539264cfdb None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:16:57.098592 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 748/1503] 10.222.0.33 () {54 vars in 963 bytes} [Thu Jun 2 19:16:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:16:57.103848 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e3964f-c6f7-4a1b-83d4-0b02a53e3d51 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:16:57.104150 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e3964f-c6f7-4a1b-83d4-0b02a53e3d51 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:16:57.104436 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e3964f-c6f7-4a1b-83d4-0b02a53e3d51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:16:57.125577 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-13e3964f-c6f7-4a1b-83d4-0b02a53e3d51 None None] MFA Rules not processed for user `091860b2577f49c5af61c75803a90c0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:16:57.126891 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-13e3964f-c6f7-4a1b-83d4-0b02a53e3d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:16:57.173733 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:16:57.181437 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 756/1504] 10.222.0.33 () {58 vars in 1060 bytes} [Thu Jun 2 19:16:57 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 19:16:57.190076 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402b880d-4ed8-4b73-8f7b-5c539264cfdb None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:16:57.200710 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402b880d-4ed8-4b73-8f7b-5c539264cfdb None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:16:57.204231 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac33c49-07aa-441c-8e28-3767897420da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:16:57.204865 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac33c49-07aa-441c-8e28-3767897420da service cinder] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:16:57.205159 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac33c49-07aa-441c-8e28-3767897420da service cinder] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:16:57.205439 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac33c49-07aa-441c-8e28-3767897420da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:16:57.213975 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac33c49-07aa-441c-8e28-3767897420da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:16:57.215486 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac33c49-07aa-441c-8e28-3767897420da service cinder] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:16:57.224725 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 749/1505] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:16:57 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:16:57.271719 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e203857-7ee4-4d43-97fb-a3cce9b4aac2 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:16:57.272009 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e203857-7ee4-4d43-97fb-a3cce9b4aac2 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:16:57.272287 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e203857-7ee4-4d43-97fb-a3cce9b4aac2 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:16:57.273574 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 757/1506] 10.222.0.33 () {52 vars in 829 bytes} [Thu Jun 2 19:16:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:16:57.277268 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac33c49-07aa-441c-8e28-3767897420da service cinder] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:16:57.339561 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac33c49-07aa-441c-8e28-3767897420da service cinder] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:16:57.344224 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a934d2-a6b1-4394-9453-9d92d487b9df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:16:57.345112 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a934d2-a6b1-4394-9453-9d92d487b9df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:16:57.345712 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a934d2-a6b1-4394-9453-9d92d487b9df admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:16:57.345712 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a934d2-a6b1-4394-9453-9d92d487b9df admin admin] PATH_INFO: `/v3/users/b5ce32a50e9d430c955c5188102e1454` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:16:57.346884 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a934d2-a6b1-4394-9453-9d92d487b9df admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5ce32a50e9d430c955c5188102e1454)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:16:57.347913 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: WARNING py.warnings [None req-f3a934d2-a6b1-4394-9453-9d92d487b9df admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 02 19:16:57.347913 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: warnings.warn(msg) Jun 02 19:16:57.347913 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: Jun 02 19:16:57.349111 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a934d2-a6b1-4394-9453-9d92d487b9df admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:16:57.569984 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-f3a934d2-a6b1-4394-9453-9d92d487b9df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5ce32a50e9d430c955c5188102e1454'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:16:57.582666 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-f3a934d2-a6b1-4394-9453-9d92d487b9df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5ce32a50e9d430c955c5188102e1454'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:16:57.583456 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-f3a934d2-a6b1-4394-9453-9d92d487b9df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5ce32a50e9d430c955c5188102e1454'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:16:57.636500 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-f3a934d2-a6b1-4394-9453-9d92d487b9df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5ce32a50e9d430c955c5188102e1454'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:16:57.637226 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-f3a934d2-a6b1-4394-9453-9d92d487b9df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5ce32a50e9d430c955c5188102e1454'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:16:57.657127 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:16:57.665886 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 750/1507] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:16:57 2022] DELETE /identity/v3/users/b5ce32a50e9d430c955c5188102e1454 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:16:57.671076 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c40d935-ea2c-4517-9ecd-a4d84c512aac None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:16:57.671579 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c40d935-ea2c-4517-9ecd-a4d84c512aac None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:16:57.671686 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c40d935-ea2c-4517-9ecd-a4d84c512aac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:16:57.689374 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-6c40d935-ea2c-4517-9ecd-a4d84c512aac None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:16:57.690624 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-6c40d935-ea2c-4517-9ecd-a4d84c512aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:16:57.757037 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 758/1508] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:16:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:16:57.767717 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a934d2-a6b1-4394-9453-9d92d487b9df admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:16:57.769159 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-f3a934d2-a6b1-4394-9453-9d92d487b9df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:16:57.838160 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a934d2-a6b1-4394-9453-9d92d487b9df admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:16:57.842334 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c523eae-2562-42d5-8be0-e52934fb38d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:16:57.843050 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c523eae-2562-42d5-8be0-e52934fb38d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:16:57.843338 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c523eae-2562-42d5-8be0-e52934fb38d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:16:57.843606 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c523eae-2562-42d5-8be0-e52934fb38d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:16:57.853916 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c523eae-2562-42d5-8be0-e52934fb38d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:16:57.855433 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c523eae-2562-42d5-8be0-e52934fb38d3 service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:16:57.867054 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 751/1509] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:16:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:16:58.370347 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c40d935-ea2c-4517-9ecd-a4d84c512aac None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:16:58.375549 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-6c40d935-ea2c-4517-9ecd-a4d84c512aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:16:58.384227 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c40d935-ea2c-4517-9ecd-a4d84c512aac None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:16:58.389773 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b21939-e1ac-4ee9-a5b8-e21e158d3572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:16:58.390650 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b21939-e1ac-4ee9-a5b8-e21e158d3572 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:16:58.390991 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b21939-e1ac-4ee9-a5b8-e21e158d3572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:16:58.391269 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b21939-e1ac-4ee9-a5b8-e21e158d3572 admin admin] PATH_INFO: `/v3/projects/6f74fd99317d48cb86cbb51c6b33fe8d` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:16:58.392479 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b21939-e1ac-4ee9-a5b8-e21e158d3572 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f74fd99317d48cb86cbb51c6b33fe8d)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:16:58.393519 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: WARNING py.warnings [None req-91b21939-e1ac-4ee9-a5b8-e21e158d3572 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 02 19:16:58.393519 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: warnings.warn(msg) Jun 02 19:16:58.393519 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: Jun 02 19:16:58.394721 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b21939-e1ac-4ee9-a5b8-e21e158d3572 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:16:58.511830 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-91b21939-e1ac-4ee9-a5b8-e21e158d3572 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f74fd99317d48cb86cbb51c6b33fe8d', 'request_id': 'req-91b21939-e1ac-4ee9-a5b8-e21e158d3572'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:16:58.573885 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-91b21939-e1ac-4ee9-a5b8-e21e158d3572 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f74fd99317d48cb86cbb51c6b33fe8d', 'request_id': 'req-91b21939-e1ac-4ee9-a5b8-e21e158d3572'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:16:58.581826 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-91b21939-e1ac-4ee9-a5b8-e21e158d3572 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f74fd99317d48cb86cbb51c6b33fe8d', 'request_id': 'req-91b21939-e1ac-4ee9-a5b8-e21e158d3572'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:16:58.600196 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-91b21939-e1ac-4ee9-a5b8-e21e158d3572 admin admin] The token cache is being invalidate because project 6f74fd99317d48cb86cbb51c6b33fe8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:16:58.600493 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-91b21939-e1ac-4ee9-a5b8-e21e158d3572 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:16:58.601317 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-91b21939-e1ac-4ee9-a5b8-e21e158d3572 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:16:58.602986 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 759/1510] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:16:58 2022] DELETE /identity/v3/projects/6f74fd99317d48cb86cbb51c6b33fe8d => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:17:07.861933 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:17:08.596959 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:18:36.930979 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2f14e0-15a7-4bb9-8535-b460193947ee None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:18:36.931576 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2f14e0-15a7-4bb9-8535-b460193947ee None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:18:36.931576 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2f14e0-15a7-4bb9-8535-b460193947ee None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:18:36.932816 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 752/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 2 19:18:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:18:36.938247 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f85447-fc41-4f40-8e49-8706dcb0c62b None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:18:36.938716 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f85447-fc41-4f40-8e49-8706dcb0c62b None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:18:36.938854 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f85447-fc41-4f40-8e49-8706dcb0c62b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:18:36.957407 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-c9f85447-fc41-4f40-8e49-8706dcb0c62b None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:18:36.958614 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-c9f85447-fc41-4f40-8e49-8706dcb0c62b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:18:37.020117 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 760/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 2 19:18:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:18:37.189578 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2f14e0-15a7-4bb9-8535-b460193947ee None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:18:37.191413 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-ed2f14e0-15a7-4bb9-8535-b460193947ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:18:37.266183 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2f14e0-15a7-4bb9-8535-b460193947ee None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:18:37.271849 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f127783-f908-4da5-a5a5-996966ca52c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:18:37.272529 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f127783-f908-4da5-a5a5-996966ca52c2 service nova] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:18:37.272816 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f127783-f908-4da5-a5a5-996966ca52c2 service nova] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:18:37.273095 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f127783-f908-4da5-a5a5-996966ca52c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:18:37.282587 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f127783-f908-4da5-a5a5-996966ca52c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:18:37.284287 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f127783-f908-4da5-a5a5-996966ca52c2 service nova] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:18:37.295135 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 753/1513] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:18:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:18:38.182436 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a98d17d-7bb2-4d3c-bd81-823395c03496 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:18:38.183030 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a98d17d-7bb2-4d3c-bd81-823395c03496 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:18:38.183030 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a98d17d-7bb2-4d3c-bd81-823395c03496 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:18:38.184421 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 761/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 2 19:18:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:18:38.189940 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc849930-708f-4241-90e6-13dc8e0b5c2d None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:18:38.190372 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc849930-708f-4241-90e6-13dc8e0b5c2d None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:18:38.190504 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc849930-708f-4241-90e6-13dc8e0b5c2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:18:38.208889 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-dc849930-708f-4241-90e6-13dc8e0b5c2d None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:18:38.210102 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-dc849930-708f-4241-90e6-13dc8e0b5c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:18:38.218680 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 754/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 2 19:18:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:18:38.377664 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a98d17d-7bb2-4d3c-bd81-823395c03496 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:18:38.392593 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a98d17d-7bb2-4d3c-bd81-823395c03496 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:18:38.397960 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63a1ab-2d35-499f-a487-202a54041be3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:18:38.398609 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63a1ab-2d35-499f-a487-202a54041be3 service nova] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:18:38.402095 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63a1ab-2d35-499f-a487-202a54041be3 service nova] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:18:38.402095 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63a1ab-2d35-499f-a487-202a54041be3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:18:38.407771 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e63a1ab-2d35-499f-a487-202a54041be3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:18:38.409380 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e63a1ab-2d35-499f-a487-202a54041be3 service nova] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:18:38.419061 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 762/1516] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:18:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:18:39.544082 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74531635-de1b-4220-94d0-5950663ad506 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:18:39.544663 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74531635-de1b-4220-94d0-5950663ad506 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:18:39.544663 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74531635-de1b-4220-94d0-5950663ad506 None None] PATH_INFO: `/v3` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:18:39.545975 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 755/1517] 10.222.0.32 () {54 vars in 925 bytes} [Thu Jun 2 19:18:39 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:18:39.552152 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0439312d-86aa-462c-aca2-88adebb0e3ed None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:18:39.552608 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0439312d-86aa-462c-aca2-88adebb0e3ed None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:18:39.552714 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0439312d-86aa-462c-aca2-88adebb0e3ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:18:39.572299 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-0439312d-86aa-462c-aca2-88adebb0e3ed None None] MFA Rules not processed for user `73ae837e99764bebbe7a6e06ac9049e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:18:39.573517 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-0439312d-86aa-462c-aca2-88adebb0e3ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:18:39.637232 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 763/1518] 10.222.0.32 () {58 vars in 1015 bytes} [Thu Jun 2 19:18:39 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:18:44.194712 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a10ae4-6421-4840-9eb0-b212430ce4dc None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:18:44.195284 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a10ae4-6421-4840-9eb0-b212430ce4dc None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:18:44.195411 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a10ae4-6421-4840-9eb0-b212430ce4dc None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:18:44.196679 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 756/1519] 10.222.0.33 () {54 vars in 960 bytes} [Thu Jun 2 19:18:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:18:44.202949 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3045e20-d395-4a89-b510-1c307bc1a67a None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:18:44.203405 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3045e20-d395-4a89-b510-1c307bc1a67a None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:18:44.203513 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3045e20-d395-4a89-b510-1c307bc1a67a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:18:44.222041 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-a3045e20-d395-4a89-b510-1c307bc1a67a None None] MFA Rules not processed for user `73ae837e99764bebbe7a6e06ac9049e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:18:44.223295 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-a3045e20-d395-4a89-b510-1c307bc1a67a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:18:44.239452 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 764/1520] 10.222.0.33 () {58 vars in 1057 bytes} [Thu Jun 2 19:18:44 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:18:44.247251 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a10ae4-6421-4840-9eb0-b212430ce4dc None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:18:44.261960 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a10ae4-6421-4840-9eb0-b212430ce4dc None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:18:44.268567 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2f9ccb-0916-43d4-bc97-519f184adb32 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73ae837e99764bebbe7a6e06ac9049e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:18:44.269224 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f9ccb-0916-43d4-bc97-519f184adb32 service placement] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:18:44.269526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f9ccb-0916-43d4-bc97-519f184adb32 service placement] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:18:44.269800 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f9ccb-0916-43d4-bc97-519f184adb32 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:18:44.278782 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2f9ccb-0916-43d4-bc97-519f184adb32 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:18:44.280958 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2f9ccb-0916-43d4-bc97-519f184adb32 service placement] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:18:44.291764 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 757/1521] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:18:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:18:49.740718 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c27cb5-5db1-43ab-8434-3f224e2659fb None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:18:49.741386 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c27cb5-5db1-43ab-8434-3f224e2659fb None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:18:49.741386 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c27cb5-5db1-43ab-8434-3f224e2659fb None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:18:49.742657 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 765/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 2 19:18:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:18:49.747970 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264b3d64-afa5-4938-832b-ef74cf36f2ab None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:18:49.749318 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264b3d64-afa5-4938-832b-ef74cf36f2ab None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:18:49.749498 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264b3d64-afa5-4938-832b-ef74cf36f2ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:18:49.768634 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-264b3d64-afa5-4938-832b-ef74cf36f2ab None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:18:49.770163 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-264b3d64-afa5-4938-832b-ef74cf36f2ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:18:49.778071 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 758/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 2 19:18:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:18:49.933175 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c27cb5-5db1-43ab-8434-3f224e2659fb None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:18:49.948405 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c27cb5-5db1-43ab-8434-3f224e2659fb None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:18:49.954679 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb48ee71-5bac-40f2-8481-2576064e13d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:18:49.955513 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48ee71-5bac-40f2-8481-2576064e13d2 service nova] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:18:49.955708 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48ee71-5bac-40f2-8481-2576064e13d2 service nova] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:18:49.955988 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48ee71-5bac-40f2-8481-2576064e13d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:18:49.964846 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb48ee71-5bac-40f2-8481-2576064e13d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:18:49.966449 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb48ee71-5bac-40f2-8481-2576064e13d2 service nova] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:18:49.976139 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 766/1524] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:18:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:18:50.875131 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06825213-e362-41d7-a97e-de426c77c665 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:18:50.875665 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06825213-e362-41d7-a97e-de426c77c665 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:18:50.875665 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06825213-e362-41d7-a97e-de426c77c665 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:18:50.876987 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 759/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 2 19:18:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:18:50.881862 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee1167-d471-4f46-88b3-b1fa404dad04 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:18:50.882298 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee1167-d471-4f46-88b3-b1fa404dad04 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:18:50.882401 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee1167-d471-4f46-88b3-b1fa404dad04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:18:50.899959 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-f6ee1167-d471-4f46-88b3-b1fa404dad04 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:18:50.901146 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-f6ee1167-d471-4f46-88b3-b1fa404dad04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:18:50.908831 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 767/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 2 19:18:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:18:51.067269 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06825213-e362-41d7-a97e-de426c77c665 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:18:51.081423 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06825213-e362-41d7-a97e-de426c77c665 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:18:51.086860 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58256df1-a3f0-4476-9748-df6b8924ddbe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:18:51.087499 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58256df1-a3f0-4476-9748-df6b8924ddbe service nova] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:18:51.087787 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58256df1-a3f0-4476-9748-df6b8924ddbe service nova] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:18:51.088070 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58256df1-a3f0-4476-9748-df6b8924ddbe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:18:51.096670 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58256df1-a3f0-4476-9748-df6b8924ddbe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:18:51.098248 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58256df1-a3f0-4476-9748-df6b8924ddbe service nova] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:18:51.107667 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 760/1527] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:18:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:18:55.395745 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1579914e-99b6-4956-a025-3e01a99888de None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:18:55.396257 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1579914e-99b6-4956-a025-3e01a99888de None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:18:55.396257 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1579914e-99b6-4956-a025-3e01a99888de None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:18:55.397605 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 768/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 2 19:18:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:18:55.403456 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da3dfc3-0407-4598-b17f-5b7986c7ce51 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:18:55.404235 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da3dfc3-0407-4598-b17f-5b7986c7ce51 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:18:55.404522 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da3dfc3-0407-4598-b17f-5b7986c7ce51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:18:55.422833 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-8da3dfc3-0407-4598-b17f-5b7986c7ce51 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:18:55.424038 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-8da3dfc3-0407-4598-b17f-5b7986c7ce51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:18:55.431897 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 761/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 2 19:18:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:18:55.469180 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1579914e-99b6-4956-a025-3e01a99888de None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:18:55.471036 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-1579914e-99b6-4956-a025-3e01a99888de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:18:55.529705 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1579914e-99b6-4956-a025-3e01a99888de None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:18:55.533468 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48b5032-d2a5-4b8a-b17a-c35955a80312 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:18:55.534119 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48b5032-d2a5-4b8a-b17a-c35955a80312 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:18:55.534409 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48b5032-d2a5-4b8a-b17a-c35955a80312 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:18:55.534685 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48b5032-d2a5-4b8a-b17a-c35955a80312 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:18:55.543826 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48b5032-d2a5-4b8a-b17a-c35955a80312 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:18:55.545382 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48b5032-d2a5-4b8a-b17a-c35955a80312 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:18:55.550668 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:18:55.555373 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 769/1530] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:18:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:18:57.839744 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7671af7-c464-4f29-9419-12752084481f None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:18:57.840314 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7671af7-c464-4f29-9419-12752084481f None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:18:57.840314 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7671af7-c464-4f29-9419-12752084481f None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:18:57.841684 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 762/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 2 19:18:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:18:57.847667 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd8e907-ba85-4d5e-a32e-be1f2be72ff0 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:18:57.848071 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd8e907-ba85-4d5e-a32e-be1f2be72ff0 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:18:57.848238 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd8e907-ba85-4d5e-a32e-be1f2be72ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:18:57.867197 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-0dd8e907-ba85-4d5e-a32e-be1f2be72ff0 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:18:57.868420 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-0dd8e907-ba85-4d5e-a32e-be1f2be72ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:18:57.876494 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 770/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 2 19:18:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:18:57.912537 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7671af7-c464-4f29-9419-12752084481f None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:18:57.923913 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7671af7-c464-4f29-9419-12752084481f None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:18:57.927825 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900f5fbd-66f7-4fe6-87f7-92207868d644 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:18:57.928466 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f5fbd-66f7-4fe6-87f7-92207868d644 service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:18:57.928756 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f5fbd-66f7-4fe6-87f7-92207868d644 service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:18:57.929040 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f5fbd-66f7-4fe6-87f7-92207868d644 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:18:57.935031 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=58 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:18:57.938704 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900f5fbd-66f7-4fe6-87f7-92207868d644 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:18:57.940245 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900f5fbd-66f7-4fe6-87f7-92207868d644 service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:18:57.950025 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 763/1533] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:18:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:19:01.804941 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639477c8-5a35-4995-af78-a7eb45950271 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:19:01.805505 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639477c8-5a35-4995-af78-a7eb45950271 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:19:01.805505 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639477c8-5a35-4995-af78-a7eb45950271 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:19:01.806920 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 771/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 2 19:19:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:19:01.813019 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a73ef3-c1d2-48b3-b0f1-b19967e036ff None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:19:01.813478 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a73ef3-c1d2-48b3-b0f1-b19967e036ff None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:19:01.813590 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a73ef3-c1d2-48b3-b0f1-b19967e036ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:19:01.831933 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-b0a73ef3-c1d2-48b3-b0f1-b19967e036ff None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:19:01.833162 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-b0a73ef3-c1d2-48b3-b0f1-b19967e036ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:19:01.841061 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 764/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 2 19:19:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:19:02.122107 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639477c8-5a35-4995-af78-a7eb45950271 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:19:02.123919 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-639477c8-5a35-4995-af78-a7eb45950271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:19:02.181647 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639477c8-5a35-4995-af78-a7eb45950271 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:19:02.185259 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f6a844-ba6e-4e7d-b4e0-1de22f059f3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:19:02.185909 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f6a844-ba6e-4e7d-b4e0-1de22f059f3a service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:19:02.186193 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f6a844-ba6e-4e7d-b4e0-1de22f059f3a service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:19:02.186515 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f6a844-ba6e-4e7d-b4e0-1de22f059f3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:19:02.195702 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f6a844-ba6e-4e7d-b4e0-1de22f059f3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:19:02.197146 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f6a844-ba6e-4e7d-b4e0-1de22f059f3a service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:19:02.207160 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 772/1536] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:19:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:19:04.539569 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b4fb07-f0fb-42c2-91ea-a512bed37de9 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:19:04.540166 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b4fb07-f0fb-42c2-91ea-a512bed37de9 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:19:04.540166 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b4fb07-f0fb-42c2-91ea-a512bed37de9 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:19:04.541506 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 765/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 2 19:19:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:19:04.547404 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b9378-7dd9-4d2e-ab45-079cc45ef99b None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:19:04.547844 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b9378-7dd9-4d2e-ab45-079cc45ef99b None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:19:04.547973 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b9378-7dd9-4d2e-ab45-079cc45ef99b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:19:04.567202 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-504b9378-7dd9-4d2e-ab45-079cc45ef99b None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:19:04.568414 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-504b9378-7dd9-4d2e-ab45-079cc45ef99b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:19:04.576697 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 773/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 2 19:19:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:19:04.847811 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b4fb07-f0fb-42c2-91ea-a512bed37de9 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:19:04.849523 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-b0b4fb07-f0fb-42c2-91ea-a512bed37de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:19:04.863192 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b4fb07-f0fb-42c2-91ea-a512bed37de9 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:19:04.866973 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003de8f5-3a25-4ec0-a725-c4f20215c704 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:19:04.867619 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003de8f5-3a25-4ec0-a725-c4f20215c704 service cinder] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:19:04.867907 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003de8f5-3a25-4ec0-a725-c4f20215c704 service cinder] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:19:04.868187 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003de8f5-3a25-4ec0-a725-c4f20215c704 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:19:04.877125 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003de8f5-3a25-4ec0-a725-c4f20215c704 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:19:04.878562 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003de8f5-3a25-4ec0-a725-c4f20215c704 service cinder] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:19:04.888025 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 766/1539] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:19:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:19:14.560550 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=32 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:19:14.883879 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=12 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:20:37.865008 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e90eaed-02a3-41e5-8a92-a2a344d07430 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:37.865937 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e90eaed-02a3-41e5-8a92-a2a344d07430 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:37.865937 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e90eaed-02a3-41e5-8a92-a2a344d07430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:37.895998 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-8e90eaed-02a3-41e5-8a92-a2a344d07430 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:20:37.897247 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-8e90eaed-02a3-41e5-8a92-a2a344d07430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:37.902710 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b228a-6aba-41be-8a18-0fd979e95d75 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:37.903288 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b228a-6aba-41be-8a18-0fd979e95d75 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:37.903385 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b228a-6aba-41be-8a18-0fd979e95d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:37.910510 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 774/1540] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:20:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:20:37.928269 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962afe26-4bc3-49dd-a868-ba108e273e8c None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:37.928610 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962afe26-4bc3-49dd-a868-ba108e273e8c None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:37.928852 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962afe26-4bc3-49dd-a868-ba108e273e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:37.935788 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-641b228a-6aba-41be-8a18-0fd979e95d75 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:20:37.937018 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-641b228a-6aba-41be-8a18-0fd979e95d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:37.955511 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 767/1541] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:20:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:20:37.960102 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-962afe26-4bc3-49dd-a868-ba108e273e8c None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:20:37.961589 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-962afe26-4bc3-49dd-a868-ba108e273e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:37.964194 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b2535-eff6-4265-b796-ff8e41d90660 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:37.964497 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b2535-eff6-4265-b796-ff8e41d90660 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:37.964758 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b2535-eff6-4265-b796-ff8e41d90660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:37.979621 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 775/1542] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:20:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 19:20:37.984457 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962afe26-4bc3-49dd-a868-ba108e273e8c None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:38.001185 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962afe26-4bc3-49dd-a868-ba108e273e8c None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:38.003884 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-c81b2535-eff6-4265-b796-ff8e41d90660 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:20:38.005111 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-c81b2535-eff6-4265-b796-ff8e41d90660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:38.012815 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2e174f-8a0b-4132-b3e8-724c213d380a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:38.013509 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e174f-8a0b-4132-b3e8-724c213d380a admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:38.013798 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e174f-8a0b-4132-b3e8-724c213d380a admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:38.014085 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e174f-8a0b-4132-b3e8-724c213d380a admin admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:38.014876 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2e174f-8a0b-4132-b3e8-724c213d380a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:20:38.015197 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2e174f-8a0b-4132-b3e8-724c213d380a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:38.016859 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2e174f-8a0b-4132-b3e8-724c213d380a admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:38.020184 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 768/1543] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:20:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 19:20:38.024812 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81b2535-eff6-4265-b796-ff8e41d90660 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:38.042237 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 776/1544] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 2 19:20:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:20:38.043333 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81b2535-eff6-4265-b796-ff8e41d90660 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:38.045038 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2e174f-8a0b-4132-b3e8-724c213d380a admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:38.056196 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052e09f6-9602-43d3-9bec-77116a2b50b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:38.056884 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052e09f6-9602-43d3-9bec-77116a2b50b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:38.057165 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052e09f6-9602-43d3-9bec-77116a2b50b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:38.057442 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052e09f6-9602-43d3-9bec-77116a2b50b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:38.058259 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052e09f6-9602-43d3-9bec-77116a2b50b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:20:38.058555 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052e09f6-9602-43d3-9bec-77116a2b50b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:38.059638 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: WARNING py.warnings [None req-052e09f6-9602-43d3-9bec-77116a2b50b3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 02 19:20:38.059638 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: warnings.warn(msg) Jun 02 19:20:38.059638 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: Jun 02 19:20:38.060528 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052e09f6-9602-43d3-9bec-77116a2b50b3 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:38.063354 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2e174f-8a0b-4132-b3e8-724c213d380a admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:38.083834 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e65e72-80f5-46eb-8761-0054f8ec06bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:38.084525 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e65e72-80f5-46eb-8761-0054f8ec06bd admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:38.084820 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e65e72-80f5-46eb-8761-0054f8ec06bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:38.085097 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e65e72-80f5-46eb-8761-0054f8ec06bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:38.085212 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 769/1545] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 2 19:20:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:20:38.085820 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e65e72-80f5-46eb-8761-0054f8ec06bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:38.086972 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: WARNING py.warnings [None req-26e65e72-80f5-46eb-8761-0054f8ec06bd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 02 19:20:38.086972 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: warnings.warn(msg) Jun 02 19:20:38.086972 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: Jun 02 19:20:38.088180 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e65e72-80f5-46eb-8761-0054f8ec06bd admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:38.093423 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052e09f6-9602-43d3-9bec-77116a2b50b3 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:38.109383 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052e09f6-9602-43d3-9bec-77116a2b50b3 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:38.143639 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed6ce68-c0bb-4bba-8fe7-e868ae19e62e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:38.144345 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed6ce68-c0bb-4bba-8fe7-e868ae19e62e admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:38.144638 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed6ce68-c0bb-4bba-8fe7-e868ae19e62e admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:38.144913 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed6ce68-c0bb-4bba-8fe7-e868ae19e62e admin admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:38.145634 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed6ce68-c0bb-4bba-8fe7-e868ae19e62e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:38.148062 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed6ce68-c0bb-4bba-8fe7-e868ae19e62e admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:38.216923 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 777/1546] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:20:38 2022] POST /identity/v3/projects => generated 391 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 02 19:20:38.221665 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e65e72-80f5-46eb-8761-0054f8ec06bd admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:38.266724 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 770/1547] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:20:38 2022] POST /identity/v3/projects => generated 375 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 02 19:20:38.268116 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7eb1d5-14b9-4d05-9943-6526cffc6705 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:38.268405 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7eb1d5-14b9-4d05-9943-6526cffc6705 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:38.268684 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7eb1d5-14b9-4d05-9943-6526cffc6705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:38.304385 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-3a7eb1d5-14b9-4d05-9943-6526cffc6705 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:20:38.306029 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-3a7eb1d5-14b9-4d05-9943-6526cffc6705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:38.314094 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e65e72-80f5-46eb-8761-0054f8ec06bd admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:38.390996 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 771/1548] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:20:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:20:38.391776 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7eb1d5-14b9-4d05-9943-6526cffc6705 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:38.400259 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511ceca0-e841-4fd6-87bb-cb7355a9ef5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:38.400955 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511ceca0-e841-4fd6-87bb-cb7355a9ef5d admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:38.401246 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511ceca0-e841-4fd6-87bb-cb7355a9ef5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:38.401526 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511ceca0-e841-4fd6-87bb-cb7355a9ef5d admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:38.402089 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7eb1d5-14b9-4d05-9943-6526cffc6705 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:38.402218 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511ceca0-e841-4fd6-87bb-cb7355a9ef5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:38.404196 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511ceca0-e841-4fd6-87bb-cb7355a9ef5d admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:38.422824 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05475917-65ee-4cf2-9e17-8952cf1b3a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:38.424074 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05475917-65ee-4cf2-9e17-8952cf1b3a5d admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:38.424366 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05475917-65ee-4cf2-9e17-8952cf1b3a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:38.424649 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05475917-65ee-4cf2-9e17-8952cf1b3a5d admin admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:38.427780 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05475917-65ee-4cf2-9e17-8952cf1b3a5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:38.428840 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: WARNING py.warnings [None req-05475917-65ee-4cf2-9e17-8952cf1b3a5d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 02 19:20:38.428840 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: warnings.warn(msg) Jun 02 19:20:38.428840 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: Jun 02 19:20:38.430068 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05475917-65ee-4cf2-9e17-8952cf1b3a5d admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:38.485397 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 778/1549] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:20:38 2022] POST /identity/v3/users => generated 296 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 02 19:20:38.487601 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d32741-1686-47e2-9ca3-e5548efb9cf3 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:38.489158 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d32741-1686-47e2-9ca3-e5548efb9cf3 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:38.489158 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d32741-1686-47e2-9ca3-e5548efb9cf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:38.511561 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 772/1550] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:20:38 2022] POST /identity/v3/users => generated 288 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:20:38.513019 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0019f6-2d5d-4fa2-a104-77cdd97eb56c None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:38.513320 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0019f6-2d5d-4fa2-a104-77cdd97eb56c None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:38.513788 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0019f6-2d5d-4fa2-a104-77cdd97eb56c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:38.514495 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-a7d32741-1686-47e2-9ca3-e5548efb9cf3 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:20:38.516129 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-a7d32741-1686-47e2-9ca3-e5548efb9cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:38.531664 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 779/1551] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:20:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:20:38.534809 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-7e0019f6-2d5d-4fa2-a104-77cdd97eb56c None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:20:38.536032 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-7e0019f6-2d5d-4fa2-a104-77cdd97eb56c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:38.538045 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301c5faa-726f-42e0-ae8c-878f6386f555 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:38.538381 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301c5faa-726f-42e0-ae8c-878f6386f555 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:38.539044 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301c5faa-726f-42e0-ae8c-878f6386f555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:38.544688 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 773/1552] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:20:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:20:38.546255 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcca0f3b-5004-437b-bf30-39359f6b428e None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:38.546403 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcca0f3b-5004-437b-bf30-39359f6b428e None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:38.546657 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcca0f3b-5004-437b-bf30-39359f6b428e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:38.558511 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-301c5faa-726f-42e0-ae8c-878f6386f555 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:20:38.559792 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-301c5faa-726f-42e0-ae8c-878f6386f555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:38.566160 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-bcca0f3b-5004-437b-bf30-39359f6b428e None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:20:38.567373 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-bcca0f3b-5004-437b-bf30-39359f6b428e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:38.568785 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 780/1553] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:20:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:20:38.569957 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301c5faa-726f-42e0-ae8c-878f6386f555 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:38.579990 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301c5faa-726f-42e0-ae8c-878f6386f555 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:38.582448 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 774/1554] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:20:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:20:38.584090 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2605da7d-6db1-4e0f-9b1a-e4feb38f92ab None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:38.584377 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2605da7d-6db1-4e0f-9b1a-e4feb38f92ab None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:38.584652 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2605da7d-6db1-4e0f-9b1a-e4feb38f92ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:38.587581 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5cf06c-b6be-4c5f-b763-728801d82289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:38.588280 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5cf06c-b6be-4c5f-b763-728801d82289 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:38.588553 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5cf06c-b6be-4c5f-b763-728801d82289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:38.588837 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5cf06c-b6be-4c5f-b763-728801d82289 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:38.589609 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5cf06c-b6be-4c5f-b763-728801d82289 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:38.593528 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5cf06c-b6be-4c5f-b763-728801d82289 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:38.603843 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-2605da7d-6db1-4e0f-9b1a-e4feb38f92ab None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:20:38.605108 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-2605da7d-6db1-4e0f-9b1a-e4feb38f92ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:38.605301 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 781/1555] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:20:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:20:38.606237 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5cf06c-b6be-4c5f-b763-728801d82289 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:38.614928 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 775/1556] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:20:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:20:38.616521 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f965340-ae16-4a36-86a0-4e6c3fef2133 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:38.616843 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f965340-ae16-4a36-86a0-4e6c3fef2133 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:38.617114 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f965340-ae16-4a36-86a0-4e6c3fef2133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:38.622943 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5cf06c-b6be-4c5f-b763-728801d82289 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:38.628712 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86ffcb2-a140-4667-926e-d2d4e2535acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:38.629393 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86ffcb2-a140-4667-926e-d2d4e2535acc admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:38.629677 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86ffcb2-a140-4667-926e-d2d4e2535acc admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:38.629953 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86ffcb2-a140-4667-926e-d2d4e2535acc admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:38.630691 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86ffcb2-a140-4667-926e-d2d4e2535acc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:38.632399 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86ffcb2-a140-4667-926e-d2d4e2535acc admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:38.639227 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-9f965340-ae16-4a36-86a0-4e6c3fef2133 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:20:38.640441 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-9f965340-ae16-4a36-86a0-4e6c3fef2133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:38.643249 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 782/1557] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:20:38 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:20:38.644522 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa8b1a3-4924-44f8-9e5b-a6485f60f245 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:38.644811 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa8b1a3-4924-44f8-9e5b-a6485f60f245 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:38.645089 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa8b1a3-4924-44f8-9e5b-a6485f60f245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:38.652733 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 776/1558] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:20:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:20:38.653578 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f965340-ae16-4a36-86a0-4e6c3fef2133 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:38.664421 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f965340-ae16-4a36-86a0-4e6c3fef2133 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:38.667409 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-caa8b1a3-4924-44f8-9e5b-a6485f60f245 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:20:38.668626 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-caa8b1a3-4924-44f8-9e5b-a6485f60f245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:38.669259 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d737ad22-30bb-411d-9989-29cf435d38e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:38.669935 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d737ad22-30bb-411d-9989-29cf435d38e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:38.670217 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d737ad22-30bb-411d-9989-29cf435d38e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:38.670504 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d737ad22-30bb-411d-9989-29cf435d38e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:38.671284 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d737ad22-30bb-411d-9989-29cf435d38e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:20:38.671589 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d737ad22-30bb-411d-9989-29cf435d38e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:38.673257 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d737ad22-30bb-411d-9989-29cf435d38e2 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:38.677552 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 783/1559] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:20:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:20:38.682617 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d293fec0-5eea-43f4-b492-2c6983bd89ed None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:38.682939 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d293fec0-5eea-43f4-b492-2c6983bd89ed None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:38.683222 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d293fec0-5eea-43f4-b492-2c6983bd89ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:38.687119 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 777/1560] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 2 19:20:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:20:38.688426 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2d4d0a-959c-44f5-9acf-031cf228176b None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:38.688715 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2d4d0a-959c-44f5-9acf-031cf228176b None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:38.688998 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2d4d0a-959c-44f5-9acf-031cf228176b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:38.701243 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:20:38.707965 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-fe2d4d0a-959c-44f5-9acf-031cf228176b None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:20:38.712437 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-d293fec0-5eea-43f4-b492-2c6983bd89ed None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:20:38.712639 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-fe2d4d0a-959c-44f5-9acf-031cf228176b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:38.712745 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-d293fec0-5eea-43f4-b492-2c6983bd89ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:38.717500 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 778/1561] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:20:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:20:38.718462 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 784/1562] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:20:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:20:38.718604 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2d4d0a-959c-44f5-9acf-031cf228176b None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:38.719595 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60256e54-5ee4-4a9f-8f81-e17a8a302287 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:38.719887 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60256e54-5ee4-4a9f-8f81-e17a8a302287 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:38.720171 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60256e54-5ee4-4a9f-8f81-e17a8a302287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:38.730377 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2d4d0a-959c-44f5-9acf-031cf228176b None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:38.735632 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e5011c-0d54-4283-b17b-2af602cefc36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:38.736473 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e5011c-0d54-4283-b17b-2af602cefc36 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:38.736759 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e5011c-0d54-4283-b17b-2af602cefc36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:38.737036 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e5011c-0d54-4283-b17b-2af602cefc36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:38.737766 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e5011c-0d54-4283-b17b-2af602cefc36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:38.739442 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e5011c-0d54-4283-b17b-2af602cefc36 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:38.740868 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-60256e54-5ee4-4a9f-8f81-e17a8a302287 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:20:38.742819 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-60256e54-5ee4-4a9f-8f81-e17a8a302287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:38.750319 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 779/1563] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:20:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:20:38.751236 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e5011c-0d54-4283-b17b-2af602cefc36 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:38.752169 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 785/1564] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:20:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:20:38.754053 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0427854c-7e05-4938-8e59-79d2c84266ba None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:38.754522 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0427854c-7e05-4938-8e59-79d2c84266ba None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:38.754849 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0427854c-7e05-4938-8e59-79d2c84266ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:38.762138 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e5011c-0d54-4283-b17b-2af602cefc36 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:38.766803 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93336c59-7f65-482c-9871-010978f557eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:38.768048 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93336c59-7f65-482c-9871-010978f557eb admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:38.768333 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93336c59-7f65-482c-9871-010978f557eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:38.768611 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93336c59-7f65-482c-9871-010978f557eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:38.769345 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93336c59-7f65-482c-9871-010978f557eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:38.770985 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93336c59-7f65-482c-9871-010978f557eb admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:38.775662 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-0427854c-7e05-4938-8e59-79d2c84266ba None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:20:38.777320 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-0427854c-7e05-4938-8e59-79d2c84266ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:38.781209 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 780/1565] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:20:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:20:38.782877 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93336c59-7f65-482c-9871-010978f557eb admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:38.786362 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 786/1566] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:20:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:20:38.787208 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0427854c-7e05-4938-8e59-79d2c84266ba None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:38.794115 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93336c59-7f65-482c-9871-010978f557eb admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:38.800051 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7991c763-5d35-463c-b404-0924042032c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:38.800608 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0427854c-7e05-4938-8e59-79d2c84266ba None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:38.801087 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7991c763-5d35-463c-b404-0924042032c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:38.801526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7991c763-5d35-463c-b404-0924042032c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:38.801904 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7991c763-5d35-463c-b404-0924042032c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:38.802885 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7991c763-5d35-463c-b404-0924042032c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:20:38.803406 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7991c763-5d35-463c-b404-0924042032c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:38.805399 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7991c763-5d35-463c-b404-0924042032c1 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:38.805788 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffd2fc4-9a80-4ac5-99b4-e39572e0098b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:38.806453 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd2fc4-9a80-4ac5-99b4-e39572e0098b admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:38.806732 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd2fc4-9a80-4ac5-99b4-e39572e0098b admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:38.807064 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd2fc4-9a80-4ac5-99b4-e39572e0098b admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:38.807729 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffd2fc4-9a80-4ac5-99b4-e39572e0098b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:38.809623 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffd2fc4-9a80-4ac5-99b4-e39572e0098b admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:38.820057 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 781/1567] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 2 19:20:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:20:38.821170 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7991c763-5d35-463c-b404-0924042032c1 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:38.832063 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7991c763-5d35-463c-b404-0924042032c1 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:38.836884 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6918f3c-e607-4f7f-8db7-7d7033af5b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:38.837561 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6918f3c-e607-4f7f-8db7-7d7033af5b7d admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:38.837844 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6918f3c-e607-4f7f-8db7-7d7033af5b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:38.838124 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6918f3c-e607-4f7f-8db7-7d7033af5b7d admin admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:38.838897 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6918f3c-e607-4f7f-8db7-7d7033af5b7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:20:38.839210 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6918f3c-e607-4f7f-8db7-7d7033af5b7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:38.840834 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6918f3c-e607-4f7f-8db7-7d7033af5b7d admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:38.851069 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:20:38.858225 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 782/1568] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 2 19:20:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:20:38.858480 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6918f3c-e607-4f7f-8db7-7d7033af5b7d admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:38.872676 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6918f3c-e607-4f7f-8db7-7d7033af5b7d admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:38.877395 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5dcc4d-9602-48e3-bae5-17425087a9e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:38.878049 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5dcc4d-9602-48e3-bae5-17425087a9e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:38.878459 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5dcc4d-9602-48e3-bae5-17425087a9e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:38.878741 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5dcc4d-9602-48e3-bae5-17425087a9e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:38.879510 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5dcc4d-9602-48e3-bae5-17425087a9e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:20:38.879819 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5dcc4d-9602-48e3-bae5-17425087a9e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:38.881768 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5dcc4d-9602-48e3-bae5-17425087a9e0 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:38.898408 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 783/1569] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 2 19:20:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:20:38.898408 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5dcc4d-9602-48e3-bae5-17425087a9e0 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:38.906117 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5dcc4d-9602-48e3-bae5-17425087a9e0 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:38.911401 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836d1b57-7592-41e9-bf07-91893cd63fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:38.912309 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836d1b57-7592-41e9-bf07-91893cd63fd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:38.912595 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836d1b57-7592-41e9-bf07-91893cd63fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:38.912877 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836d1b57-7592-41e9-bf07-91893cd63fd8 admin admin] PATH_INFO: `/v3/projects/016b97603d7948ff8ca42cb48a09b6b6/users/7f950b5cc5fa44c988b945e9cd553874/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:38.928126 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836d1b57-7592-41e9-bf07-91893cd63fd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=016b97603d7948ff8ca42cb48a09b6b6, user_id=7f950b5cc5fa44c988b945e9cd553874, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:38.934673 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836d1b57-7592-41e9-bf07-91893cd63fd8 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:38.934830 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 787/1570] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:20:38 2022] POST /identity/v3/projects => generated 373 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:20:38.934830 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffd2fc4-9a80-4ac5-99b4-e39572e0098b admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:38.999194 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffd2fc4-9a80-4ac5-99b4-e39572e0098b admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:39.004099 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b50222d-41c7-4c71-9b2d-0d8e7fede487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:39.004783 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b50222d-41c7-4c71-9b2d-0d8e7fede487 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:39.005074 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b50222d-41c7-4c71-9b2d-0d8e7fede487 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:39.005360 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b50222d-41c7-4c71-9b2d-0d8e7fede487 admin admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:39.006099 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b50222d-41c7-4c71-9b2d-0d8e7fede487 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:20:39.006414 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b50222d-41c7-4c71-9b2d-0d8e7fede487 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:39.008062 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b50222d-41c7-4c71-9b2d-0d8e7fede487 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:39.021000 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 788/1571] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 2 19:20:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:20:39.022040 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b50222d-41c7-4c71-9b2d-0d8e7fede487 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:39.032114 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b50222d-41c7-4c71-9b2d-0d8e7fede487 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:39.037383 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602919f4-8e11-40a2-a9a8-85fc9d0ebcf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:39.038278 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602919f4-8e11-40a2-a9a8-85fc9d0ebcf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:39.038559 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602919f4-8e11-40a2-a9a8-85fc9d0ebcf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:39.038868 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602919f4-8e11-40a2-a9a8-85fc9d0ebcf2 admin admin] PATH_INFO: `/v3/projects/549cbca4cab7489f830e0dfc8976c8bd/users/1021580c951947b0829957aa3ffe318a/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:39.044363 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 784/1572] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:20:38 2022] PUT /identity/v3/projects/016b97603d7948ff8ca42cb48a09b6b6/users/7f950b5cc5fa44c988b945e9cd553874/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:20:39.045469 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836d1b57-7592-41e9-bf07-91893cd63fd8 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:39.055308 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602919f4-8e11-40a2-a9a8-85fc9d0ebcf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=549cbca4cab7489f830e0dfc8976c8bd, user_id=1021580c951947b0829957aa3ffe318a, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:39.056405 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: WARNING py.warnings [None req-602919f4-8e11-40a2-a9a8-85fc9d0ebcf2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 02 19:20:39.056405 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: warnings.warn(msg) Jun 02 19:20:39.056405 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: Jun 02 19:20:39.058373 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602919f4-8e11-40a2-a9a8-85fc9d0ebcf2 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:39.109713 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836d1b57-7592-41e9-bf07-91893cd63fd8 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:39.114379 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e017fd02-aaf2-4cb2-9643-213827a28206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:39.115154 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e017fd02-aaf2-4cb2-9643-213827a28206 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:39.115772 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e017fd02-aaf2-4cb2-9643-213827a28206 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:39.116054 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e017fd02-aaf2-4cb2-9643-213827a28206 admin admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:39.116796 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e017fd02-aaf2-4cb2-9643-213827a28206 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:20:39.117103 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e017fd02-aaf2-4cb2-9643-213827a28206 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:39.118708 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e017fd02-aaf2-4cb2-9643-213827a28206 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:39.131399 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 785/1573] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 2 19:20:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:20:39.132222 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e017fd02-aaf2-4cb2-9643-213827a28206 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:39.144176 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e017fd02-aaf2-4cb2-9643-213827a28206 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:39.149001 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38f37ec-5128-42b6-b4b7-fc0e224d8e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:39.149611 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38f37ec-5128-42b6-b4b7-fc0e224d8e86 admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:39.149899 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38f37ec-5128-42b6-b4b7-fc0e224d8e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:39.150185 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38f37ec-5128-42b6-b4b7-fc0e224d8e86 admin admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:39.150922 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38f37ec-5128-42b6-b4b7-fc0e224d8e86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:39.152831 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38f37ec-5128-42b6-b4b7-fc0e224d8e86 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:39.152970 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 789/1574] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:20:39 2022] PUT /identity/v3/projects/549cbca4cab7489f830e0dfc8976c8bd/users/1021580c951947b0829957aa3ffe318a/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:20:39.153687 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602919f4-8e11-40a2-a9a8-85fc9d0ebcf2 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:39.227020 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602919f4-8e11-40a2-a9a8-85fc9d0ebcf2 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:39.232038 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de53956d-4e63-48a6-a49d-6475d195b615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:39.234954 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de53956d-4e63-48a6-a49d-6475d195b615 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:39.234954 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de53956d-4e63-48a6-a49d-6475d195b615 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:39.234954 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de53956d-4e63-48a6-a49d-6475d195b615 admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:39.234954 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de53956d-4e63-48a6-a49d-6475d195b615 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:39.236146 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de53956d-4e63-48a6-a49d-6475d195b615 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:39.269662 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 786/1575] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:20:39 2022] POST /identity/v3/projects => generated 385 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:20:39.270682 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38f37ec-5128-42b6-b4b7-fc0e224d8e86 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:39.333014 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38f37ec-5128-42b6-b4b7-fc0e224d8e86 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:39.337853 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee88b2e-97e9-460a-8194-74b7aa5af6c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:39.338516 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee88b2e-97e9-460a-8194-74b7aa5af6c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:39.338830 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee88b2e-97e9-460a-8194-74b7aa5af6c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:39.339208 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee88b2e-97e9-460a-8194-74b7aa5af6c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:39.339898 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee88b2e-97e9-460a-8194-74b7aa5af6c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:39.341824 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee88b2e-97e9-460a-8194-74b7aa5af6c1 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:39.365044 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 790/1576] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:20:39 2022] POST /identity/v3/projects => generated 379 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:20:39.366425 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de53956d-4e63-48a6-a49d-6475d195b615 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:39.389101 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:20:39.431486 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de53956d-4e63-48a6-a49d-6475d195b615 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:39.436135 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b62765-78cf-48f8-a1f2-4fae266f16bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:39.436929 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b62765-78cf-48f8-a1f2-4fae266f16bd admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:39.451279 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b62765-78cf-48f8-a1f2-4fae266f16bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:39.451279 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b62765-78cf-48f8-a1f2-4fae266f16bd admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:39.451279 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b62765-78cf-48f8-a1f2-4fae266f16bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:39.451279 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b62765-78cf-48f8-a1f2-4fae266f16bd admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:39.451672 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 787/1577] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:20:39 2022] POST /identity/v3/projects => generated 373 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:20:39.451672 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee88b2e-97e9-460a-8194-74b7aa5af6c1 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:39.472415 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:20:39.512473 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 791/1578] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:20:39 2022] POST /identity/v3/users => generated 287 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:20:39.514167 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62246db0-f676-4371-ade8-63c01c3271bc None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:39.514456 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62246db0-f676-4371-ade8-63c01c3271bc None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:39.514731 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62246db0-f676-4371-ade8-63c01c3271bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:39.519218 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee88b2e-97e9-460a-8194-74b7aa5af6c1 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:39.525722 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598eb448-c7fb-4ef5-bf0d-9d45a917d463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:39.526516 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598eb448-c7fb-4ef5-bf0d-9d45a917d463 admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:39.526832 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598eb448-c7fb-4ef5-bf0d-9d45a917d463 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:39.527621 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598eb448-c7fb-4ef5-bf0d-9d45a917d463 admin admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:39.528400 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598eb448-c7fb-4ef5-bf0d-9d45a917d463 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:39.530312 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598eb448-c7fb-4ef5-bf0d-9d45a917d463 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:39.532682 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-62246db0-f676-4371-ade8-63c01c3271bc None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:20:39.533942 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-62246db0-f676-4371-ade8-63c01c3271bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:39.541739 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 792/1579] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:20:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:20:39.545077 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62246db0-f676-4371-ade8-63c01c3271bc None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:39.553449 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62246db0-f676-4371-ade8-63c01c3271bc None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:39.558151 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c66f4cb-ad73-4ec4-9a13-3c34f93c9a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:39.558847 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c66f4cb-ad73-4ec4-9a13-3c34f93c9a76 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:39.559139 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c66f4cb-ad73-4ec4-9a13-3c34f93c9a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:39.559422 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c66f4cb-ad73-4ec4-9a13-3c34f93c9a76 admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:39.560488 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c66f4cb-ad73-4ec4-9a13-3c34f93c9a76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:39.562133 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c66f4cb-ad73-4ec4-9a13-3c34f93c9a76 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:39.594172 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 788/1580] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:20:39 2022] POST /identity/v3/projects => generated 377 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:20:39.595626 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd2869-717c-4e00-ad9d-96c40e5bfb8f None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:39.596002 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd2869-717c-4e00-ad9d-96c40e5bfb8f None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:39.596356 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd2869-717c-4e00-ad9d-96c40e5bfb8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:39.623805 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-cccd2869-717c-4e00-ad9d-96c40e5bfb8f None None] MFA Rules not processed for user `1021580c951947b0829957aa3ffe318a`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:20:39.625071 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-cccd2869-717c-4e00-ad9d-96c40e5bfb8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:39.637404 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 793/1581] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:20:39 2022] POST /identity/v3/projects => generated 367 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:20:39.637404 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c66f4cb-ad73-4ec4-9a13-3c34f93c9a76 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:39.695195 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 789/1582] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:20:39 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:20:39.695985 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccd2869-717c-4e00-ad9d-96c40e5bfb8f None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:39.713860 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c66f4cb-ad73-4ec4-9a13-3c34f93c9a76 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:39.721289 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0d1d2f-87db-4d6a-8dbd-285fd9ee4c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:39.722146 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0d1d2f-87db-4d6a-8dbd-285fd9ee4c10 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:39.722537 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0d1d2f-87db-4d6a-8dbd-285fd9ee4c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:39.723402 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0d1d2f-87db-4d6a-8dbd-285fd9ee4c10 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:39.724220 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0d1d2f-87db-4d6a-8dbd-285fd9ee4c10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:39.726773 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0d1d2f-87db-4d6a-8dbd-285fd9ee4c10 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:39.764536 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccd2869-717c-4e00-ad9d-96c40e5bfb8f None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:39.769193 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54b7e4b-968b-45d8-9ce7-b489950f3e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:39.769872 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54b7e4b-968b-45d8-9ce7-b489950f3e2a admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:39.770156 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54b7e4b-968b-45d8-9ce7-b489950f3e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:39.770434 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54b7e4b-968b-45d8-9ce7-b489950f3e2a admin admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:39.771170 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54b7e4b-968b-45d8-9ce7-b489950f3e2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:39.773085 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54b7e4b-968b-45d8-9ce7-b489950f3e2a admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:39.810436 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 794/1583] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:20:39 2022] POST /identity/v3/users => generated 293 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:20:39.811414 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0d1d2f-87db-4d6a-8dbd-285fd9ee4c10 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:39.822696 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0d1d2f-87db-4d6a-8dbd-285fd9ee4c10 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:39.827763 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0899e1-af06-4275-9fca-f6523b6900b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:39.828466 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0899e1-af06-4275-9fca-f6523b6900b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:39.828751 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0899e1-af06-4275-9fca-f6523b6900b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:39.829028 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0899e1-af06-4275-9fca-f6523b6900b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:39.829737 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0899e1-af06-4275-9fca-f6523b6900b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:39.831906 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0899e1-af06-4275-9fca-f6523b6900b5 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:39.890479 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 790/1584] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:20:39 2022] POST /identity/v3/users => generated 290 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:20:39.891386 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54b7e4b-968b-45d8-9ce7-b489950f3e2a admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:39.901141 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54b7e4b-968b-45d8-9ce7-b489950f3e2a admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:39.905728 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1ea893-3e5d-4e8c-9cca-310d696665f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:39.906370 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ea893-3e5d-4e8c-9cca-310d696665f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:39.906944 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ea893-3e5d-4e8c-9cca-310d696665f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:39.906944 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ea893-3e5d-4e8c-9cca-310d696665f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:39.907663 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1ea893-3e5d-4e8c-9cca-310d696665f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:39.909816 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1ea893-3e5d-4e8c-9cca-310d696665f9 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:39.918773 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 795/1585] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:20:39 2022] POST /identity/v3/users => generated 287 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:20:39.921941 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0899e1-af06-4275-9fca-f6523b6900b5 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:39.922058 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 791/1586] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:20:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:20:39.922058 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1ea893-3e5d-4e8c-9cca-310d696665f9 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:39.932202 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1ea893-3e5d-4e8c-9cca-310d696665f9 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:39.936847 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8c774d-2dcb-47f0-a6d0-29a1d96e7f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:39.937519 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8c774d-2dcb-47f0-a6d0-29a1d96e7f10 admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:39.937805 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8c774d-2dcb-47f0-a6d0-29a1d96e7f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:39.938088 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8c774d-2dcb-47f0-a6d0-29a1d96e7f10 admin admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:39.938850 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8c774d-2dcb-47f0-a6d0-29a1d96e7f10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:39.941180 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8c774d-2dcb-47f0-a6d0-29a1d96e7f10 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:39.951307 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:20:39.978650 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0899e1-af06-4275-9fca-f6523b6900b5 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:39.982271 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4467ff3a-a814-4ab6-b675-e164c4022373 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:39.982961 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4467ff3a-a814-4ab6-b675-e164c4022373 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:39.983249 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4467ff3a-a814-4ab6-b675-e164c4022373 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:39.983526 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4467ff3a-a814-4ab6-b675-e164c4022373 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:39.991365 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:20:39.993600 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4467ff3a-a814-4ab6-b675-e164c4022373 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:39.995086 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4467ff3a-a814-4ab6-b675-e164c4022373 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:40.004397 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 796/1587] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:20:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:20:40.005163 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4467ff3a-a814-4ab6-b675-e164c4022373 service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:40.013911 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4467ff3a-a814-4ab6-b675-e164c4022373 service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:40.018687 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7105c0-6f04-4375-a20e-1eaedd46ba0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:40.023496 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7105c0-6f04-4375-a20e-1eaedd46ba0a admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:40.023496 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7105c0-6f04-4375-a20e-1eaedd46ba0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:40.023496 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7105c0-6f04-4375-a20e-1eaedd46ba0a admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:40.023496 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7105c0-6f04-4375-a20e-1eaedd46ba0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:40.023496 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7105c0-6f04-4375-a20e-1eaedd46ba0a admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:40.050868 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 792/1588] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:20:39 2022] POST /identity/v3/users => generated 289 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:20:40.052337 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e126ff-dda8-41ba-ad9d-1a3f240b0af2 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:40.052630 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e126ff-dda8-41ba-ad9d-1a3f240b0af2 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:40.052910 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e126ff-dda8-41ba-ad9d-1a3f240b0af2 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:40.054299 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 793/1589] 10.222.0.33 () {54 vars in 961 bytes} [Thu Jun 2 19:20:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:20:40.055030 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e126ff-dda8-41ba-ad9d-1a3f240b0af2 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:40.067458 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e126ff-dda8-41ba-ad9d-1a3f240b0af2 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:40.072248 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de67829-4fd7-4f5e-acde-c7a82cae068d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:40.072898 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de67829-4fd7-4f5e-acde-c7a82cae068d admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:40.073186 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de67829-4fd7-4f5e-acde-c7a82cae068d admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:40.073469 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de67829-4fd7-4f5e-acde-c7a82cae068d admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:40.074195 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de67829-4fd7-4f5e-acde-c7a82cae068d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:40.075820 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de67829-4fd7-4f5e-acde-c7a82cae068d admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:40.086500 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 794/1590] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:20:40.087517 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de67829-4fd7-4f5e-acde-c7a82cae068d admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:40.097346 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de67829-4fd7-4f5e-acde-c7a82cae068d admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:40.101985 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8b5b3f-6a08-4fee-a7c1-8d8ffb457631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:40.102622 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b5b3f-6a08-4fee-a7c1-8d8ffb457631 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:40.102944 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b5b3f-6a08-4fee-a7c1-8d8ffb457631 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:40.103231 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b5b3f-6a08-4fee-a7c1-8d8ffb457631 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:40.103941 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8b5b3f-6a08-4fee-a7c1-8d8ffb457631 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:40.105546 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8b5b3f-6a08-4fee-a7c1-8d8ffb457631 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:40.116522 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 795/1591] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:20:40.121183 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8b5b3f-6a08-4fee-a7c1-8d8ffb457631 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:40.127226 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8b5b3f-6a08-4fee-a7c1-8d8ffb457631 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:40.131858 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9d85bd-715f-438e-b3cc-6d12a56b392b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:40.132513 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9d85bd-715f-438e-b3cc-6d12a56b392b admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:40.132801 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9d85bd-715f-438e-b3cc-6d12a56b392b admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:40.133081 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9d85bd-715f-438e-b3cc-6d12a56b392b admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:40.133844 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9d85bd-715f-438e-b3cc-6d12a56b392b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:40.135463 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9d85bd-715f-438e-b3cc-6d12a56b392b admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:40.145842 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 796/1592] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:20:40.146653 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9d85bd-715f-438e-b3cc-6d12a56b392b admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:40.153975 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 797/1593] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:20:40 2022] POST /identity/v3/users => generated 284 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:20:40.155638 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7105c0-6f04-4375-a20e-1eaedd46ba0a admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:40.159388 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9d85bd-715f-438e-b3cc-6d12a56b392b admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:40.164254 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8336d838-e531-4d33-b348-1f5f16e860d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:40.164993 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8336d838-e531-4d33-b348-1f5f16e860d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:40.165341 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8336d838-e531-4d33-b348-1f5f16e860d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:40.165679 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8336d838-e531-4d33-b348-1f5f16e860d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:40.166475 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8336d838-e531-4d33-b348-1f5f16e860d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:40.167206 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7105c0-6f04-4375-a20e-1eaedd46ba0a admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:40.168314 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8336d838-e531-4d33-b348-1f5f16e860d0 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:40.172316 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23439c89-ccff-48c5-ac3a-8997e890a64e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:40.173131 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23439c89-ccff-48c5-ac3a-8997e890a64e admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:40.173554 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23439c89-ccff-48c5-ac3a-8997e890a64e admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:40.174422 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23439c89-ccff-48c5-ac3a-8997e890a64e admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:40.175701 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23439c89-ccff-48c5-ac3a-8997e890a64e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:40.177359 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23439c89-ccff-48c5-ac3a-8997e890a64e admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:40.179598 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 797/1594] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:20:40.180939 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b25a621-5efb-4078-bedd-ff2eb49e736f None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:40.181292 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b25a621-5efb-4078-bedd-ff2eb49e736f None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:40.181642 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b25a621-5efb-4078-bedd-ff2eb49e736f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:40.192753 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 798/1595] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:20:40.193863 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23439c89-ccff-48c5-ac3a-8997e890a64e admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:40.205670 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23439c89-ccff-48c5-ac3a-8997e890a64e admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:40.206430 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-2b25a621-5efb-4078-bedd-ff2eb49e736f None None] MFA Rules not processed for user `70664d5cbdee47ee89c120761939cc75`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:20:40.207665 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-2b25a621-5efb-4078-bedd-ff2eb49e736f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:40.211337 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe5c500-26e0-4fce-b99f-22562348b50a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:40.213121 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe5c500-26e0-4fce-b99f-22562348b50a admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:40.213121 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe5c500-26e0-4fce-b99f-22562348b50a admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:40.213121 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe5c500-26e0-4fce-b99f-22562348b50a admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:40.213865 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe5c500-26e0-4fce-b99f-22562348b50a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:40.215610 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe5c500-26e0-4fce-b99f-22562348b50a admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:40.227113 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 799/1596] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:20:40.227909 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe5c500-26e0-4fce-b99f-22562348b50a admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:40.237686 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe5c500-26e0-4fce-b99f-22562348b50a admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:40.248010 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224beaea-d2c4-40c0-8791-29fad0893cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:40.248010 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224beaea-d2c4-40c0-8791-29fad0893cf2 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:40.248010 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224beaea-d2c4-40c0-8791-29fad0893cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:40.248010 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224beaea-d2c4-40c0-8791-29fad0893cf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:40.248573 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224beaea-d2c4-40c0-8791-29fad0893cf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:40.248573 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224beaea-d2c4-40c0-8791-29fad0893cf2 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:40.256751 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 800/1597] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:20:40.259731 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224beaea-d2c4-40c0-8791-29fad0893cf2 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:40.270456 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224beaea-d2c4-40c0-8791-29fad0893cf2 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:40.275604 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892217cf-cade-480c-a0bf-d4a5322c4e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:40.276285 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892217cf-cade-480c-a0bf-d4a5322c4e17 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:40.276598 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892217cf-cade-480c-a0bf-d4a5322c4e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:40.276880 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892217cf-cade-480c-a0bf-d4a5322c4e17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:40.277720 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892217cf-cade-480c-a0bf-d4a5322c4e17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:40.279279 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892217cf-cade-480c-a0bf-d4a5322c4e17 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:40.289881 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 798/1598] 10.222.0.33 () {58 vars in 1058 bytes} [Thu Jun 2 19:20:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:20:40.291814 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b25a621-5efb-4078-bedd-ff2eb49e736f None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:40.292971 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 801/1599] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:20:40.293823 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892217cf-cade-480c-a0bf-d4a5322c4e17 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:40.304738 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892217cf-cade-480c-a0bf-d4a5322c4e17 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:40.306963 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b25a621-5efb-4078-bedd-ff2eb49e736f None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:40.309580 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2058936-b60f-41e1-a98b-2f47b6d9d2b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:40.310698 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2058936-b60f-41e1-a98b-2f47b6d9d2b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:40.310849 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2058936-b60f-41e1-a98b-2f47b6d9d2b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:40.311117 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2058936-b60f-41e1-a98b-2f47b6d9d2b6 admin admin] PATH_INFO: `/v3/projects/b2c95d85cb07426ab0bf3f1d8dfd27c8/users/3520bf40daa4423b915ea8bcaec447fd/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:40.311839 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5f429d-7469-4d5f-adae-ad0d4e825134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:40.312622 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5f429d-7469-4d5f-adae-ad0d4e825134 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:40.312787 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5f429d-7469-4d5f-adae-ad0d4e825134 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:40.313067 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5f429d-7469-4d5f-adae-ad0d4e825134 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:40.313828 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5f429d-7469-4d5f-adae-ad0d4e825134 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:40.315520 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5f429d-7469-4d5f-adae-ad0d4e825134 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:40.326310 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 799/1600] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:20:40.327221 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5f429d-7469-4d5f-adae-ad0d4e825134 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:40.332711 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2058936-b60f-41e1-a98b-2f47b6d9d2b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2c95d85cb07426ab0bf3f1d8dfd27c8, user_id=3520bf40daa4423b915ea8bcaec447fd, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:40.335513 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2058936-b60f-41e1-a98b-2f47b6d9d2b6 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:40.339231 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5f429d-7469-4d5f-adae-ad0d4e825134 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:40.344281 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2933c14-f336-4c3c-8d6d-22cf0eda9c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:40.348063 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2933c14-f336-4c3c-8d6d-22cf0eda9c23 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:40.348063 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2933c14-f336-4c3c-8d6d-22cf0eda9c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:40.348063 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2933c14-f336-4c3c-8d6d-22cf0eda9c23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:40.348063 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2933c14-f336-4c3c-8d6d-22cf0eda9c23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:40.348632 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2933c14-f336-4c3c-8d6d-22cf0eda9c23 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:40.359320 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 800/1601] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:20:40.360213 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2933c14-f336-4c3c-8d6d-22cf0eda9c23 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:40.372870 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2933c14-f336-4c3c-8d6d-22cf0eda9c23 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:40.377784 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b99efd-2de5-468f-8399-bdba1a695e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:40.378680 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b99efd-2de5-468f-8399-bdba1a695e0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:40.379004 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b99efd-2de5-468f-8399-bdba1a695e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:40.379352 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b99efd-2de5-468f-8399-bdba1a695e0b admin admin] PATH_INFO: `/v3/projects/f582628f76674cb9821d1cbf11aaec7d/users/a0681f3eff324262b4289bf6d8f4c311/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:40.403166 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b99efd-2de5-468f-8399-bdba1a695e0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f582628f76674cb9821d1cbf11aaec7d, user_id=a0681f3eff324262b4289bf6d8f4c311, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:40.406889 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b99efd-2de5-468f-8399-bdba1a695e0b admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:40.429330 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 802/1602] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:20:40 2022] PUT /identity/v3/projects/b2c95d85cb07426ab0bf3f1d8dfd27c8/users/3520bf40daa4423b915ea8bcaec447fd/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:20:40.430178 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2058936-b60f-41e1-a98b-2f47b6d9d2b6 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:40.475334 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 801/1603] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:20:40 2022] PUT /identity/v3/projects/f582628f76674cb9821d1cbf11aaec7d/users/a0681f3eff324262b4289bf6d8f4c311/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:20:40.476518 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b99efd-2de5-468f-8399-bdba1a695e0b admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:40.522470 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2058936-b60f-41e1-a98b-2f47b6d9d2b6 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:40.583253 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:20:40.588396 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b99efd-2de5-468f-8399-bdba1a695e0b admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:40.594171 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098517f2-52a6-46c4-9e7a-d38476fe7955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:40.595469 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098517f2-52a6-46c4-9e7a-d38476fe7955 admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:40.595969 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098517f2-52a6-46c4-9e7a-d38476fe7955 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:40.596107 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcc2c6d-4acf-4ff6-8623-95d4e7f43d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:40.596579 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098517f2-52a6-46c4-9e7a-d38476fe7955 admin admin] PATH_INFO: `/v3/projects/2490a4964da74cf1837be4c8843ad906/users/0424095acdf048cc8ebaef0499abff9b/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:40.596999 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc2c6d-4acf-4ff6-8623-95d4e7f43d60 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:40.597286 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc2c6d-4acf-4ff6-8623-95d4e7f43d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:40.597564 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc2c6d-4acf-4ff6-8623-95d4e7f43d60 admin admin] PATH_INFO: `/v3/projects/738f5dc47a184fd2962b9895a3827dd3/users/c68940b1a426492eb94d0a9fc0ea0c3b/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:40.613795 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcc2c6d-4acf-4ff6-8623-95d4e7f43d60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=738f5dc47a184fd2962b9895a3827dd3, user_id=c68940b1a426492eb94d0a9fc0ea0c3b, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:40.614465 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098517f2-52a6-46c4-9e7a-d38476fe7955 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2490a4964da74cf1837be4c8843ad906, user_id=0424095acdf048cc8ebaef0499abff9b, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:40.616935 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcc2c6d-4acf-4ff6-8623-95d4e7f43d60 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:40.618259 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098517f2-52a6-46c4-9e7a-d38476fe7955 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:40.686326 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:20:40.694511 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 802/1604] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:20:40 2022] PUT /identity/v3/projects/2490a4964da74cf1837be4c8843ad906/users/0424095acdf048cc8ebaef0499abff9b/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:20:40.695139 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098517f2-52a6-46c4-9e7a-d38476fe7955 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:40.702228 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 803/1605] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:20:40 2022] PUT /identity/v3/projects/738f5dc47a184fd2962b9895a3827dd3/users/c68940b1a426492eb94d0a9fc0ea0c3b/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:20:40.703419 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcc2c6d-4acf-4ff6-8623-95d4e7f43d60 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:40.777780 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcc2c6d-4acf-4ff6-8623-95d4e7f43d60 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:40.783036 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbe58a0-ad8f-4d29-ae69-452049ee2bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:40.784064 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe58a0-ad8f-4d29-ae69-452049ee2bfb admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:40.784529 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe58a0-ad8f-4d29-ae69-452049ee2bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:40.784798 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe58a0-ad8f-4d29-ae69-452049ee2bfb admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:40.785597 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbe58a0-ad8f-4d29-ae69-452049ee2bfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:40.787455 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098517f2-52a6-46c4-9e7a-d38476fe7955 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:40.787594 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbe58a0-ad8f-4d29-ae69-452049ee2bfb admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:40.798709 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 804/1606] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:20:40.802672 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbe58a0-ad8f-4d29-ae69-452049ee2bfb admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:40.812791 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbe58a0-ad8f-4d29-ae69-452049ee2bfb admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:40.825779 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b1b0f0-13ac-46c2-89aa-9f4b3015fff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:40.832011 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b1b0f0-13ac-46c2-89aa-9f4b3015fff0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:40.832426 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b1b0f0-13ac-46c2-89aa-9f4b3015fff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:40.832704 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b1b0f0-13ac-46c2-89aa-9f4b3015fff0 admin admin] PATH_INFO: `/v3/projects/626d49f321144516b095148aa0f632dc/users/df62e9ab4d044dcaab00c5d76416e3e1/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:40.851907 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b1b0f0-13ac-46c2-89aa-9f4b3015fff0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=626d49f321144516b095148aa0f632dc, user_id=df62e9ab4d044dcaab00c5d76416e3e1, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:40.861889 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b1b0f0-13ac-46c2-89aa-9f4b3015fff0 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:40.865977 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfae5c32-b9e1-40b7-a328-5605786a40ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:40.866785 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae5c32-b9e1-40b7-a328-5605786a40ae service nova] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:40.867316 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae5c32-b9e1-40b7-a328-5605786a40ae service nova] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:40.867432 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae5c32-b9e1-40b7-a328-5605786a40ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:40.935999 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfae5c32-b9e1-40b7-a328-5605786a40ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:40.937642 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfae5c32-b9e1-40b7-a328-5605786a40ae service nova] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:40.948307 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 805/1607] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:20:40 2022] PUT /identity/v3/projects/626d49f321144516b095148aa0f632dc/users/df62e9ab4d044dcaab00c5d76416e3e1/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:20:40.949632 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3e8b9e-1724-4d90-9f7b-4ddb27f25da5 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:40.949917 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3e8b9e-1724-4d90-9f7b-4ddb27f25da5 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:40.950209 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3e8b9e-1724-4d90-9f7b-4ddb27f25da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:40.957047 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 803/1608] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:20:40 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:20:40.959733 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab3b10-ea3a-4b7d-9b80-486ffe54a48d None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:40.960246 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab3b10-ea3a-4b7d-9b80-486ffe54a48d None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:40.960842 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab3b10-ea3a-4b7d-9b80-486ffe54a48d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:40.980901 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-9b3e8b9e-1724-4d90-9f7b-4ddb27f25da5 None None] MFA Rules not processed for user `3520bf40daa4423b915ea8bcaec447fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:20:40.982300 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-9b3e8b9e-1724-4d90-9f7b-4ddb27f25da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:40.988536 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-45ab3b10-ea3a-4b7d-9b80-486ffe54a48d None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:20:40.988536 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-45ab3b10-ea3a-4b7d-9b80-486ffe54a48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:41.000769 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:20:41.064720 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 804/1609] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:20:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:20:41.068102 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e8959e-96c2-467e-a1cb-dec1a3682e3a None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:41.068102 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e8959e-96c2-467e-a1cb-dec1a3682e3a None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:41.068102 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e8959e-96c2-467e-a1cb-dec1a3682e3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:41.084585 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 806/1610] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:20:40 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:20:41.086481 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0d8cba-861c-4e9a-adc0-909caa3702dc None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:41.086973 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0d8cba-861c-4e9a-adc0-909caa3702dc None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:41.087421 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0d8cba-861c-4e9a-adc0-909caa3702dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:41.110530 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-11e8959e-96c2-467e-a1cb-dec1a3682e3a None None] MFA Rules not processed for user `0424095acdf048cc8ebaef0499abff9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:20:41.110867 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-ef0d8cba-861c-4e9a-adc0-909caa3702dc None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:20:41.111848 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-11e8959e-96c2-467e-a1cb-dec1a3682e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:41.112614 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-ef0d8cba-861c-4e9a-adc0-909caa3702dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:41.122924 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 807/1611] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:20:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:20:41.124031 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0d8cba-861c-4e9a-adc0-909caa3702dc None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:41.135515 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0d8cba-861c-4e9a-adc0-909caa3702dc None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:41.141712 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597acf2a-e953-4a0d-82e3-70916b61538b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:41.143327 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597acf2a-e953-4a0d-82e3-70916b61538b admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:41.143644 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597acf2a-e953-4a0d-82e3-70916b61538b admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:41.144299 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597acf2a-e953-4a0d-82e3-70916b61538b admin admin] PATH_INFO: `/v3/projects/68e10a6c688f4084be40b2eaf52a0784/users/9ac7b875483e45ebbffc4961d766eea1/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:41.161843 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597acf2a-e953-4a0d-82e3-70916b61538b admin admin] RBAC: Authorizing `identity:create_grant(project_id=68e10a6c688f4084be40b2eaf52a0784, user_id=9ac7b875483e45ebbffc4961d766eea1, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:41.164948 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597acf2a-e953-4a0d-82e3-70916b61538b admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:41.196380 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 805/1612] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:20:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:20:41.197767 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51b6aa5-17ed-4ea6-a24d-0c89803bbcb8 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:41.202923 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51b6aa5-17ed-4ea6-a24d-0c89803bbcb8 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:41.202923 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51b6aa5-17ed-4ea6-a24d-0c89803bbcb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:41.219332 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-f51b6aa5-17ed-4ea6-a24d-0c89803bbcb8 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:20:41.227563 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-f51b6aa5-17ed-4ea6-a24d-0c89803bbcb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:41.229125 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 808/1613] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:20:41 2022] PUT /identity/v3/projects/68e10a6c688f4084be40b2eaf52a0784/users/9ac7b875483e45ebbffc4961d766eea1/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:20:41.229125 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597acf2a-e953-4a0d-82e3-70916b61538b admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:41.292243 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:20:41.317975 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 806/1614] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:20:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:20:41.320225 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597acf2a-e953-4a0d-82e3-70916b61538b admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:41.320348 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51b6aa5-17ed-4ea6-a24d-0c89803bbcb8 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:41.327362 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93872d51-ac4d-4a1d-a6a6-1437aa0a5e73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:41.328041 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93872d51-ac4d-4a1d-a6a6-1437aa0a5e73 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:41.328328 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93872d51-ac4d-4a1d-a6a6-1437aa0a5e73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:41.328686 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93872d51-ac4d-4a1d-a6a6-1437aa0a5e73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:41.329182 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51b6aa5-17ed-4ea6-a24d-0c89803bbcb8 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:41.335299 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf334dd-d8b6-4142-bd22-f349d6fb7b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:41.335975 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf334dd-d8b6-4142-bd22-f349d6fb7b9e admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:41.336261 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf334dd-d8b6-4142-bd22-f349d6fb7b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:41.336540 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf334dd-d8b6-4142-bd22-f349d6fb7b9e admin admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:41.337710 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf334dd-d8b6-4142-bd22-f349d6fb7b9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:41.339196 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf334dd-d8b6-4142-bd22-f349d6fb7b9e admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:41.343002 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93872d51-ac4d-4a1d-a6a6-1437aa0a5e73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:41.343002 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93872d51-ac4d-4a1d-a6a6-1437aa0a5e73 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:41.352669 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 809/1615] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:20:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:20:41.354505 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93872d51-ac4d-4a1d-a6a6-1437aa0a5e73 service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:41.366364 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93872d51-ac4d-4a1d-a6a6-1437aa0a5e73 service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:41.369983 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f70603-bc4f-4d14-8dbc-17485c772810 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:41.373665 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f70603-bc4f-4d14-8dbc-17485c772810 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:41.373665 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f70603-bc4f-4d14-8dbc-17485c772810 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:41.373665 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f70603-bc4f-4d14-8dbc-17485c772810 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:41.384496 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f70603-bc4f-4d14-8dbc-17485c772810 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:41.385953 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f70603-bc4f-4d14-8dbc-17485c772810 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:41.397588 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 810/1616] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:20:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:20:41.398878 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0025d0-4d80-4ae6-a373-17156922a715 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:41.399126 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0025d0-4d80-4ae6-a373-17156922a715 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:41.402084 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0025d0-4d80-4ae6-a373-17156922a715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:41.433073 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-0a0025d0-4d80-4ae6-a373-17156922a715 None None] MFA Rules not processed for user `9ac7b875483e45ebbffc4961d766eea1`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:20:41.434290 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-0a0025d0-4d80-4ae6-a373-17156922a715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:41.494738 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 807/1617] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:20:41 2022] POST /identity/v3/projects => generated 373 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:20:41.496197 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511d0bb-2e6b-44fd-8a1a-04ad17f3b704 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:41.497922 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511d0bb-2e6b-44fd-8a1a-04ad17f3b704 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:41.498355 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511d0bb-2e6b-44fd-8a1a-04ad17f3b704 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:41.501081 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 808/1618] 10.222.0.33 () {54 vars in 961 bytes} [Thu Jun 2 19:20:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:20:41.508016 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485bed73-5d57-483e-98a5-f124c9cfe67e None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:41.508402 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485bed73-5d57-483e-98a5-f124c9cfe67e None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:41.509378 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485bed73-5d57-483e-98a5-f124c9cfe67e None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:41.513841 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 809/1619] 10.222.0.33 () {54 vars in 912 bytes} [Thu Jun 2 19:20:41 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:20:41.515870 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485bed73-5d57-483e-98a5-f124c9cfe67e None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:41.541699 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 811/1620] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:20:41 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:20:41.542666 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0025d0-4d80-4ae6-a373-17156922a715 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:41.567515 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0025d0-4d80-4ae6-a373-17156922a715 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:41.570320 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:20:41.620987 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485bed73-5d57-483e-98a5-f124c9cfe67e None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:41.625364 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c3f2b6-afa4-4a97-a65e-a4a6d315cbb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:41.626213 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c3f2b6-afa4-4a97-a65e-a4a6d315cbb2 service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:41.627852 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c3f2b6-afa4-4a97-a65e-a4a6d315cbb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:41.628290 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c3f2b6-afa4-4a97-a65e-a4a6d315cbb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:41.670424 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a51f85-d887-4d5f-b446-1adca4ac870a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:41.673389 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a51f85-d887-4d5f-b446-1adca4ac870a admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:41.674445 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a51f85-d887-4d5f-b446-1adca4ac870a admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:41.674664 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a51f85-d887-4d5f-b446-1adca4ac870a admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:41.675509 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a51f85-d887-4d5f-b446-1adca4ac870a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:41.680214 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a51f85-d887-4d5f-b446-1adca4ac870a admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:41.711709 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c3f2b6-afa4-4a97-a65e-a4a6d315cbb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:41.713158 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c3f2b6-afa4-4a97-a65e-a4a6d315cbb2 service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:41.728164 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 810/1621] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:20:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:20:41.729439 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38287b-49c6-4edb-8b17-3b16d911ffec None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:41.730112 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38287b-49c6-4edb-8b17-3b16d911ffec None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:41.730669 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38287b-49c6-4edb-8b17-3b16d911ffec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:41.752983 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-fc38287b-49c6-4edb-8b17-3b16d911ffec None None] MFA Rules not processed for user `70664d5cbdee47ee89c120761939cc75`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:20:41.754206 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-fc38287b-49c6-4edb-8b17-3b16d911ffec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:41.812336 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 812/1622] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:20:41 2022] POST /identity/v3/users => generated 293 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:20:41.813283 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b19ad5-21ef-4820-ae18-81b62f854953 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:41.814137 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b19ad5-21ef-4820-ae18-81b62f854953 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:41.814137 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b19ad5-21ef-4820-ae18-81b62f854953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:41.858234 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-d9b19ad5-21ef-4820-ae18-81b62f854953 None None] MFA Rules not processed for user `70664d5cbdee47ee89c120761939cc75`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:20:41.862390 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-d9b19ad5-21ef-4820-ae18-81b62f854953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:41.870671 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 811/1623] 10.222.0.33 () {58 vars in 1058 bytes} [Thu Jun 2 19:20:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:20:41.881720 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc38287b-49c6-4edb-8b17-3b16d911ffec None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:41.893323 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 813/1624] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Jun 2 19:20:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:20:41.894724 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b19ad5-21ef-4820-ae18-81b62f854953 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:41.920357 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b19ad5-21ef-4820-ae18-81b62f854953 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:41.936288 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfa2fe9-a5d9-4f0c-acf0-3b401745ae34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:41.936288 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa2fe9-a5d9-4f0c-acf0-3b401745ae34 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:41.936288 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa2fe9-a5d9-4f0c-acf0-3b401745ae34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:41.936288 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa2fe9-a5d9-4f0c-acf0-3b401745ae34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:41.936598 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfa2fe9-a5d9-4f0c-acf0-3b401745ae34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:41.945456 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfa2fe9-a5d9-4f0c-acf0-3b401745ae34 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:41.975502 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 814/1625] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:20:41 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:20:41.976629 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfa2fe9-a5d9-4f0c-acf0-3b401745ae34 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:42.005822 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc38287b-49c6-4edb-8b17-3b16d911ffec None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:42.007194 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfa2fe9-a5d9-4f0c-acf0-3b401745ae34 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:42.011520 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a69834-43da-4bc8-bed4-7392d0ba902e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:42.012359 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a69834-43da-4bc8-bed4-7392d0ba902e service cinder] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:42.013481 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a69834-43da-4bc8-bed4-7392d0ba902e service cinder] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:42.013864 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a69834-43da-4bc8-bed4-7392d0ba902e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:42.020796 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63dff7c-6ab6-40b7-b035-c89fe37040cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:42.024972 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63dff7c-6ab6-40b7-b035-c89fe37040cc service nova] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:42.024972 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63dff7c-6ab6-40b7-b035-c89fe37040cc service nova] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:42.024972 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63dff7c-6ab6-40b7-b035-c89fe37040cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:42.043199 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:20:42.056509 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:20:42.134141 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a69834-43da-4bc8-bed4-7392d0ba902e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:42.136022 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a69834-43da-4bc8-bed4-7392d0ba902e service cinder] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:42.139316 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63dff7c-6ab6-40b7-b035-c89fe37040cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:42.146003 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63dff7c-6ab6-40b7-b035-c89fe37040cc service nova] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:42.156054 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 815/1626] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:20:41 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:20:42.157531 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63dff7c-6ab6-40b7-b035-c89fe37040cc service nova] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:42.167435 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63dff7c-6ab6-40b7-b035-c89fe37040cc service nova] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:42.171322 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118c5c2b-ab0c-4bd3-98d0-9ed084f08c5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:42.171974 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c5c2b-ab0c-4bd3-98d0-9ed084f08c5a service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:42.172259 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c5c2b-ab0c-4bd3-98d0-9ed084f08c5a service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:42.173300 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c5c2b-ab0c-4bd3-98d0-9ed084f08c5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:42.182232 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 812/1627] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:20:41 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:20:42.183553 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a69834-43da-4bc8-bed4-7392d0ba902e service cinder] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:42.188460 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118c5c2b-ab0c-4bd3-98d0-9ed084f08c5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:42.189968 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118c5c2b-ab0c-4bd3-98d0-9ed084f08c5a service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:42.210943 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 816/1628] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:20:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:20:42.218763 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a69834-43da-4bc8-bed4-7392d0ba902e service cinder] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:42.242827 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3124318-c8ea-4ce0-896e-068dff5d64b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:42.243890 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3124318-c8ea-4ce0-896e-068dff5d64b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:42.244277 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3124318-c8ea-4ce0-896e-068dff5d64b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:42.244978 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3124318-c8ea-4ce0-896e-068dff5d64b0 admin admin] PATH_INFO: `/v3/projects/2ec4f05ed28b4a89839c291b729d7920/users/3fc2f28b4a6e4e1b9fbbbe111eba5b4e/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:42.281888 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3124318-c8ea-4ce0-896e-068dff5d64b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ec4f05ed28b4a89839c291b729d7920, user_id=3fc2f28b4a6e4e1b9fbbbe111eba5b4e, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:42.290860 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3124318-c8ea-4ce0-896e-068dff5d64b0 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:42.409469 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118c5c2b-ab0c-4bd3-98d0-9ed084f08c5a service cinder] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:42.410244 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 813/1629] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:20:42 2022] PUT /identity/v3/projects/2ec4f05ed28b4a89839c291b729d7920/users/3fc2f28b4a6e4e1b9fbbbe111eba5b4e/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:20:42.425558 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118c5c2b-ab0c-4bd3-98d0-9ed084f08c5a service cinder] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:42.602395 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3124318-c8ea-4ce0-896e-068dff5d64b0 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:42.602559 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f677328-af94-4f6e-8f6c-c66010cc964d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:42.603124 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f677328-af94-4f6e-8f6c-c66010cc964d admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:42.603410 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f677328-af94-4f6e-8f6c-c66010cc964d admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:42.603690 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f677328-af94-4f6e-8f6c-c66010cc964d admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:42.604438 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f677328-af94-4f6e-8f6c-c66010cc964d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:42.606832 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f677328-af94-4f6e-8f6c-c66010cc964d admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:42.621859 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3124318-c8ea-4ce0-896e-068dff5d64b0 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:42.628152 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 817/1630] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:20:42 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:20:42.629863 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f677328-af94-4f6e-8f6c-c66010cc964d admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:42.637400 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b823940c-5891-4f1b-86e5-b0cc37c67264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:42.638795 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b823940c-5891-4f1b-86e5-b0cc37c67264 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:42.644717 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b823940c-5891-4f1b-86e5-b0cc37c67264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:42.644717 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b823940c-5891-4f1b-86e5-b0cc37c67264 admin admin] PATH_INFO: `/v3/users/1021580c951947b0829957aa3ffe318a` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:42.644717 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b823940c-5891-4f1b-86e5-b0cc37c67264 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1021580c951947b0829957aa3ffe318a)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:42.647823 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b823940c-5891-4f1b-86e5-b0cc37c67264 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:42.651923 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f677328-af94-4f6e-8f6c-c66010cc964d admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:42.673193 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ae8548-5898-4ac8-afc5-e88cf73672ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:42.673193 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ae8548-5898-4ac8-afc5-e88cf73672ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:42.673193 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ae8548-5898-4ac8-afc5-e88cf73672ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:42.673193 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ae8548-5898-4ac8-afc5-e88cf73672ba admin admin] PATH_INFO: `/v3/projects/2ec4f05ed28b4a89839c291b729d7920/users/3fc2f28b4a6e4e1b9fbbbe111eba5b4e/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:42.684332 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ae8548-5898-4ac8-afc5-e88cf73672ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ec4f05ed28b4a89839c291b729d7920, user_id=3fc2f28b4a6e4e1b9fbbbe111eba5b4e, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:42.687338 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ae8548-5898-4ac8-afc5-e88cf73672ba admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:42.709498 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 818/1631] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:20:42 2022] PUT /identity/v3/projects/2ec4f05ed28b4a89839c291b729d7920/users/3fc2f28b4a6e4e1b9fbbbe111eba5b4e/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:20:42.711302 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71de48c0-9c5e-4781-a6ed-02fcdf27cc45 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:42.711737 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71de48c0-9c5e-4781-a6ed-02fcdf27cc45 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:42.712193 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71de48c0-9c5e-4781-a6ed-02fcdf27cc45 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:42.713956 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 819/1632] 10.222.0.33 () {54 vars in 912 bytes} [Thu Jun 2 19:20:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:20:42.719586 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd090a6a-d0c8-4cc5-b454-edb0a6524ab2 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:42.720095 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd090a6a-d0c8-4cc5-b454-edb0a6524ab2 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:42.721106 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd090a6a-d0c8-4cc5-b454-edb0a6524ab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:42.755161 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-fd090a6a-d0c8-4cc5-b454-edb0a6524ab2 None None] MFA Rules not processed for user `3fc2f28b4a6e4e1b9fbbbe111eba5b4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:20:42.756756 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-fd090a6a-d0c8-4cc5-b454-edb0a6524ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:42.929411 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 820/1633] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:20:42 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 19:20:42.932325 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5988843-3b51-4514-aa28-8893f4e3a834 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:42.932325 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5988843-3b51-4514-aa28-8893f4e3a834 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:42.932325 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5988843-3b51-4514-aa28-8893f4e3a834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:42.949379 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-b823940c-5891-4f1b-86e5-b0cc37c67264 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1021580c951947b0829957aa3ffe318a'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:20:42.969390 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-c5988843-3b51-4514-aa28-8893f4e3a834 None None] MFA Rules not processed for user `70664d5cbdee47ee89c120761939cc75`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:20:42.971299 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-c5988843-3b51-4514-aa28-8893f4e3a834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:42.972323 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-b823940c-5891-4f1b-86e5-b0cc37c67264 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1021580c951947b0829957aa3ffe318a'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:20:42.973120 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-b823940c-5891-4f1b-86e5-b0cc37c67264 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1021580c951947b0829957aa3ffe318a'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:20:43.008342 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:20:43.062116 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-b823940c-5891-4f1b-86e5-b0cc37c67264 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1021580c951947b0829957aa3ffe318a'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:20:43.063272 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-b823940c-5891-4f1b-86e5-b0cc37c67264 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1021580c951947b0829957aa3ffe318a'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:20:43.110085 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 814/1634] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:20:42 2022] DELETE /identity/v3/users/1021580c951947b0829957aa3ffe318a => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:20:43.112016 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b823940c-5891-4f1b-86e5-b0cc37c67264 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:43.118601 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-b823940c-5891-4f1b-86e5-b0cc37c67264 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:43.166941 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 821/1635] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Jun 2 19:20:42 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:20:43.168826 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d8e25-285b-41c7-98eb-858c3c1ebf38 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:43.169125 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d8e25-285b-41c7-98eb-858c3c1ebf38 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:43.169411 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d8e25-285b-41c7-98eb-858c3c1ebf38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:43.211359 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-2e0d8e25-285b-41c7-98eb-858c3c1ebf38 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:20:43.214663 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-2e0d8e25-285b-41c7-98eb-858c3c1ebf38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:43.231160 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b823940c-5891-4f1b-86e5-b0cc37c67264 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:43.238727 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a075f5-1a71-4fd6-8498-38bb6725d99f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:43.241066 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a075f5-1a71-4fd6-8498-38bb6725d99f service cinder] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:43.241512 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a075f5-1a71-4fd6-8498-38bb6725d99f service cinder] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:43.242124 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a075f5-1a71-4fd6-8498-38bb6725d99f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:43.584073 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-44a075f5-1a71-4fd6-8498-38bb6725d99f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:43.584073 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:20:43.587468 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 822/1636] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:20:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:20:43.587468 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0d8e25-285b-41c7-98eb-858c3c1ebf38 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:43.587468 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-2e0d8e25-285b-41c7-98eb-858c3c1ebf38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:43.589143 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a075f5-1a71-4fd6-8498-38bb6725d99f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:43.589143 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a075f5-1a71-4fd6-8498-38bb6725d99f service cinder] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:43.589143 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 815/1637] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:20:43 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:20:43.590099 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0d8e25-285b-41c7-98eb-858c3c1ebf38 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:43.590099 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abb267e-a8aa-4798-b7c0-ce4f7f468268 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:43.590099 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb267e-a8aa-4798-b7c0-ce4f7f468268 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:43.590099 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb267e-a8aa-4798-b7c0-ce4f7f468268 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:43.591104 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb267e-a8aa-4798-b7c0-ce4f7f468268 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:43.591104 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abb267e-a8aa-4798-b7c0-ce4f7f468268 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:43.591104 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abb267e-a8aa-4798-b7c0-ce4f7f468268 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:43.591104 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 823/1638] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:20:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 02 19:20:43.683540 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a075f5-1a71-4fd6-8498-38bb6725d99f service cinder] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:43.694314 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a075f5-1a71-4fd6-8498-38bb6725d99f service cinder] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:43.700636 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0ab7f8-e91d-44b5-b737-36277597dfa5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:43.701630 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0ab7f8-e91d-44b5-b737-36277597dfa5 service cinder] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:43.702048 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0ab7f8-e91d-44b5-b737-36277597dfa5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:43.711707 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0ab7f8-e91d-44b5-b737-36277597dfa5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:43.814955 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0ab7f8-e91d-44b5-b737-36277597dfa5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:43.814955 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0ab7f8-e91d-44b5-b737-36277597dfa5 service cinder] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:43.831359 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 816/1639] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:20:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:20:44.194015 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeea371-1157-4e60-a336-a3a468dee8d5 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:44.194637 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeea371-1157-4e60-a336-a3a468dee8d5 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:44.196359 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeea371-1157-4e60-a336-a3a468dee8d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:44.262196 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-cfeea371-1157-4e60-a336-a3a468dee8d5 None None] MFA Rules not processed for user `c68940b1a426492eb94d0a9fc0ea0c3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:20:44.264813 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-cfeea371-1157-4e60-a336-a3a468dee8d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:44.399842 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:20:44.445137 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b1a48-928a-47f9-a20f-2ba0cba489a0 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:44.445657 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b1a48-928a-47f9-a20f-2ba0cba489a0 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:44.445845 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b1a48-928a-47f9-a20f-2ba0cba489a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:44.448501 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 824/1640] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:20:44 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:20:44.504714 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-a73b1a48-928a-47f9-a20f-2ba0cba489a0 None None] MFA Rules not processed for user `c68940b1a426492eb94d0a9fc0ea0c3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:20:44.506015 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-a73b1a48-928a-47f9-a20f-2ba0cba489a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:44.524597 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 817/1641] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:20:44 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:20:44.529223 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeea371-1157-4e60-a336-a3a468dee8d5 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:44.530644 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-cfeea371-1157-4e60-a336-a3a468dee8d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:44.562767 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeea371-1157-4e60-a336-a3a468dee8d5 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:44.579373 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d399014f-6f43-4249-84e0-49489e6def95 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:44.579373 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d399014f-6f43-4249-84e0-49489e6def95 service nova] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:44.579373 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d399014f-6f43-4249-84e0-49489e6def95 service nova] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:44.579645 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d399014f-6f43-4249-84e0-49489e6def95 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:44.603487 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d399014f-6f43-4249-84e0-49489e6def95 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:44.605106 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d399014f-6f43-4249-84e0-49489e6def95 service nova] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:44.627551 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 825/1642] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:20:44 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:20:44.760529 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73b1a48-928a-47f9-a20f-2ba0cba489a0 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:44.762027 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-a73b1a48-928a-47f9-a20f-2ba0cba489a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:44.785122 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73b1a48-928a-47f9-a20f-2ba0cba489a0 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:44.796695 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade2e6d6-b478-43cc-af4e-2fb5d449deca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:44.800549 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade2e6d6-b478-43cc-af4e-2fb5d449deca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:44.801296 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade2e6d6-b478-43cc-af4e-2fb5d449deca admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:44.803709 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade2e6d6-b478-43cc-af4e-2fb5d449deca admin admin] PATH_INFO: `/v3/projects/549cbca4cab7489f830e0dfc8976c8bd` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:44.805880 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade2e6d6-b478-43cc-af4e-2fb5d449deca admin admin] RBAC: Authorizing `identity:delete_project(project_id=549cbca4cab7489f830e0dfc8976c8bd)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:44.807076 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: WARNING py.warnings [None req-ade2e6d6-b478-43cc-af4e-2fb5d449deca admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 02 19:20:44.807076 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: warnings.warn(msg) Jun 02 19:20:44.807076 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: Jun 02 19:20:44.850193 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade2e6d6-b478-43cc-af4e-2fb5d449deca admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:45.094019 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-ade2e6d6-b478-43cc-af4e-2fb5d449deca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '549cbca4cab7489f830e0dfc8976c8bd', 'request_id': 'req-ade2e6d6-b478-43cc-af4e-2fb5d449deca'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:20:45.226930 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-ade2e6d6-b478-43cc-af4e-2fb5d449deca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '549cbca4cab7489f830e0dfc8976c8bd', 'request_id': 'req-ade2e6d6-b478-43cc-af4e-2fb5d449deca'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:20:45.241882 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-ade2e6d6-b478-43cc-af4e-2fb5d449deca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '549cbca4cab7489f830e0dfc8976c8bd', 'request_id': 'req-ade2e6d6-b478-43cc-af4e-2fb5d449deca'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:20:45.307143 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-ade2e6d6-b478-43cc-af4e-2fb5d449deca admin admin] The token cache is being invalidate because project 549cbca4cab7489f830e0dfc8976c8bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:20:45.307143 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-ade2e6d6-b478-43cc-af4e-2fb5d449deca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:20:45.308525 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-ade2e6d6-b478-43cc-af4e-2fb5d449deca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:20:45.310323 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 818/1643] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:20:44 2022] DELETE /identity/v3/projects/549cbca4cab7489f830e0dfc8976c8bd => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:20:45.315661 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7af7b2e-c81f-432a-a1f4-1a865a9cc48c None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:45.317447 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7af7b2e-c81f-432a-a1f4-1a865a9cc48c None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:45.318988 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7af7b2e-c81f-432a-a1f4-1a865a9cc48c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:45.351124 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-d7af7b2e-c81f-432a-a1f4-1a865a9cc48c None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:20:45.356092 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-d7af7b2e-c81f-432a-a1f4-1a865a9cc48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:45.475750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 826/1644] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:20:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:20:45.475933 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8d548-59d5-49f3-9f2e-948d24ddb89b None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:45.476710 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8d548-59d5-49f3-9f2e-948d24ddb89b None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:45.484009 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8d548-59d5-49f3-9f2e-948d24ddb89b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:45.487421 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7af7b2e-c81f-432a-a1f4-1a865a9cc48c None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:45.489992 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-d7af7b2e-c81f-432a-a1f4-1a865a9cc48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:45.512965 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7af7b2e-c81f-432a-a1f4-1a865a9cc48c None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:45.519393 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-e2c8d548-59d5-49f3-9f2e-948d24ddb89b None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:20:45.520871 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-e2c8d548-59d5-49f3-9f2e-948d24ddb89b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:45.521226 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddb8d0a-f6d6-4352-a5b0-37db74f59e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:45.523030 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddb8d0a-f6d6-4352-a5b0-37db74f59e3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:45.523554 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddb8d0a-f6d6-4352-a5b0-37db74f59e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:45.524738 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddb8d0a-f6d6-4352-a5b0-37db74f59e3b admin admin] PATH_INFO: `/v3/users/3520bf40daa4423b915ea8bcaec447fd` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:45.526374 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddb8d0a-f6d6-4352-a5b0-37db74f59e3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3520bf40daa4423b915ea8bcaec447fd)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:45.528856 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: WARNING py.warnings [None req-7ddb8d0a-f6d6-4352-a5b0-37db74f59e3b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 02 19:20:45.528856 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: warnings.warn(msg) Jun 02 19:20:45.528856 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: Jun 02 19:20:45.532221 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddb8d0a-f6d6-4352-a5b0-37db74f59e3b admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:45.542440 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 819/1645] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:20:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 19:20:45.547239 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c8d548-59d5-49f3-9f2e-948d24ddb89b None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:45.565824 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c8d548-59d5-49f3-9f2e-948d24ddb89b None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:45.579233 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6651869-7872-4a03-9a20-68a0c5da1b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:45.581615 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6651869-7872-4a03-9a20-68a0c5da1b4c admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:45.581615 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6651869-7872-4a03-9a20-68a0c5da1b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:45.581615 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6651869-7872-4a03-9a20-68a0c5da1b4c admin admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:45.582100 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6651869-7872-4a03-9a20-68a0c5da1b4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:20:45.582521 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6651869-7872-4a03-9a20-68a0c5da1b4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:45.584494 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6651869-7872-4a03-9a20-68a0c5da1b4c admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:45.605853 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:20:45.629690 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 820/1646] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 2 19:20:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:20:45.634175 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6651869-7872-4a03-9a20-68a0c5da1b4c admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:45.645317 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6651869-7872-4a03-9a20-68a0c5da1b4c admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:45.650261 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644ee98e-0a69-4993-b676-b07557bc8304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:45.651239 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ee98e-0a69-4993-b676-b07557bc8304 admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:45.651607 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ee98e-0a69-4993-b676-b07557bc8304 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:45.651960 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ee98e-0a69-4993-b676-b07557bc8304 admin admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:45.653059 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644ee98e-0a69-4993-b676-b07557bc8304 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:45.655821 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644ee98e-0a69-4993-b676-b07557bc8304 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:45.813437 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 821/1647] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:20:45 2022] POST /identity/v3/projects => generated 381 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:20:45.813437 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cf04e6-768e-42e3-9b87-3a80395bdaa2 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:45.813437 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cf04e6-768e-42e3-9b87-3a80395bdaa2 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:45.813437 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cf04e6-768e-42e3-9b87-3a80395bdaa2 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:45.813809 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 822/1648] 10.222.0.33 () {54 vars in 912 bytes} [Thu Jun 2 19:20:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:20:45.819259 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cf04e6-768e-42e3-9b87-3a80395bdaa2 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:45.832118 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-7ddb8d0a-f6d6-4352-a5b0-37db74f59e3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3520bf40daa4423b915ea8bcaec447fd'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:20:45.860186 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-7ddb8d0a-f6d6-4352-a5b0-37db74f59e3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3520bf40daa4423b915ea8bcaec447fd'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:20:45.955575 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cf04e6-768e-42e3-9b87-3a80395bdaa2 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:45.963939 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c941b-63c2-4455-b5cd-269f885786ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:45.964862 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c941b-63c2-4455-b5cd-269f885786ea admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:45.965256 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c941b-63c2-4455-b5cd-269f885786ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:45.965540 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c941b-63c2-4455-b5cd-269f885786ea admin admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:45.966311 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7c941b-63c2-4455-b5cd-269f885786ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:45.968810 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7c941b-63c2-4455-b5cd-269f885786ea admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:45.969829 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-7ddb8d0a-f6d6-4352-a5b0-37db74f59e3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3520bf40daa4423b915ea8bcaec447fd'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:20:45.971310 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-7ddb8d0a-f6d6-4352-a5b0-37db74f59e3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3520bf40daa4423b915ea8bcaec447fd'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:20:45.972688 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-7ddb8d0a-f6d6-4352-a5b0-37db74f59e3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3520bf40daa4423b915ea8bcaec447fd'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:20:46.028105 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 827/1649] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:20:45 2022] DELETE /identity/v3/users/3520bf40daa4423b915ea8bcaec447fd => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:20:46.030320 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d70e69e-2dff-41a7-a57c-e447cabc2218 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:46.030844 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d70e69e-2dff-41a7-a57c-e447cabc2218 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:46.031376 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d70e69e-2dff-41a7-a57c-e447cabc2218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:46.077804 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 823/1650] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:20:45 2022] POST /identity/v3/users => generated 291 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 02 19:20:46.082950 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e91372a-2127-489f-816e-25ff927af111 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:46.083248 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e91372a-2127-489f-816e-25ff927af111 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:46.083548 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e91372a-2127-489f-816e-25ff927af111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:46.100472 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-0d70e69e-2dff-41a7-a57c-e447cabc2218 None None] MFA Rules not processed for user `7417a54046e34e899d7b03f2f2314deb`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:20:46.102039 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-0d70e69e-2dff-41a7-a57c-e447cabc2218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:46.137300 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:20:46.140119 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-9e91372a-2127-489f-816e-25ff927af111 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:20:46.141485 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-9e91372a-2127-489f-816e-25ff927af111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:46.199584 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 828/1651] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Jun 2 19:20:46 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:20:46.202390 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d70e69e-2dff-41a7-a57c-e447cabc2218 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:46.218228 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-0d70e69e-2dff-41a7-a57c-e447cabc2218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:46.247642 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 824/1652] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:20:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:20:46.252179 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e91372a-2127-489f-816e-25ff927af111 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:46.252179 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-9e91372a-2127-489f-816e-25ff927af111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:46.266653 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e91372a-2127-489f-816e-25ff927af111 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:46.273294 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f164b7c5-e82c-4dcd-a708-0cc00f2ad887 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:46.274022 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f164b7c5-e82c-4dcd-a708-0cc00f2ad887 service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:46.274362 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f164b7c5-e82c-4dcd-a708-0cc00f2ad887 service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:46.274646 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f164b7c5-e82c-4dcd-a708-0cc00f2ad887 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:46.284437 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f164b7c5-e82c-4dcd-a708-0cc00f2ad887 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:46.286088 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f164b7c5-e82c-4dcd-a708-0cc00f2ad887 service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:46.319691 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 825/1653] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:20:46 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:20:46.320861 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f164b7c5-e82c-4dcd-a708-0cc00f2ad887 service neutron] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:46.340639 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f164b7c5-e82c-4dcd-a708-0cc00f2ad887 service neutron] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:46.353564 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad92b94-0349-41a3-9969-8cdd0678a4fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:46.355849 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad92b94-0349-41a3-9969-8cdd0678a4fb service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:46.356237 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad92b94-0349-41a3-9969-8cdd0678a4fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:46.356823 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad92b94-0349-41a3-9969-8cdd0678a4fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:46.398635 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad92b94-0349-41a3-9969-8cdd0678a4fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:46.404530 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad92b94-0349-41a3-9969-8cdd0678a4fb service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:46.439272 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 826/1654] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:20:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:20:46.442108 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d70e69e-2dff-41a7-a57c-e447cabc2218 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:46.448838 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b6ebef-d69f-4429-9c79-2aac835d763f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:46.449757 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6ebef-d69f-4429-9c79-2aac835d763f admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:46.450591 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6ebef-d69f-4429-9c79-2aac835d763f admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:46.451021 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6ebef-d69f-4429-9c79-2aac835d763f admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:46.451917 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b6ebef-d69f-4429-9c79-2aac835d763f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:46.454188 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b6ebef-d69f-4429-9c79-2aac835d763f admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:46.477655 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad92b94-0349-41a3-9969-8cdd0678a4fb service neutron] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:46.493596 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 829/1655] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:20:46 2022] GET /identity/v3/roles => generated 1420 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:20:46.495343 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad92b94-0349-41a3-9969-8cdd0678a4fb service neutron] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:46.513176 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcd002f-7721-43b8-90ec-cadaf31156c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:46.513663 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcd002f-7721-43b8-90ec-cadaf31156c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:46.513951 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcd002f-7721-43b8-90ec-cadaf31156c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:46.514231 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcd002f-7721-43b8-90ec-cadaf31156c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:46.515110 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcd002f-7721-43b8-90ec-cadaf31156c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:46.517881 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcd002f-7721-43b8-90ec-cadaf31156c2 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:46.543538 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 827/1656] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:20:46 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:20:46.548265 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b6ebef-d69f-4429-9c79-2aac835d763f admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:46.562361 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b6ebef-d69f-4429-9c79-2aac835d763f admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:46.578639 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4944200-ff52-4018-8af4-bfb6f4b13d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:46.587328 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4944200-ff52-4018-8af4-bfb6f4b13d3f admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:46.587618 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4944200-ff52-4018-8af4-bfb6f4b13d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:46.587890 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4944200-ff52-4018-8af4-bfb6f4b13d3f admin admin] PATH_INFO: `/v3/projects/99f1844d6f9b482ab843e01922db5015/users/c70eaf14c2c04d7596359dd7a1ecc16e/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:46.638023 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4944200-ff52-4018-8af4-bfb6f4b13d3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=99f1844d6f9b482ab843e01922db5015, user_id=c70eaf14c2c04d7596359dd7a1ecc16e, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:46.648225 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4944200-ff52-4018-8af4-bfb6f4b13d3f admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:46.750429 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e0c553-57f1-4a62-89ea-fd1758c4f711 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:46.750724 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e0c553-57f1-4a62-89ea-fd1758c4f711 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:46.752553 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e0c553-57f1-4a62-89ea-fd1758c4f711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:46.754362 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 830/1657] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:20:46 2022] PUT /identity/v3/projects/99f1844d6f9b482ab843e01922db5015/users/c70eaf14c2c04d7596359dd7a1ecc16e/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:20:46.831995 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-28e0c553-57f1-4a62-89ea-fd1758c4f711 None None] MFA Rules not processed for user `c70eaf14c2c04d7596359dd7a1ecc16e`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:20:46.833397 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-28e0c553-57f1-4a62-89ea-fd1758c4f711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:46.932745 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:20:47.125942 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4944200-ff52-4018-8af4-bfb6f4b13d3f admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:47.140669 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 828/1658] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:20:46 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 391 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:20:47.243846 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4944200-ff52-4018-8af4-bfb6f4b13d3f admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:47.251810 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348ac76b-e8b2-4c8d-bec0-ecaeaf23a05a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:47.254782 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ac76b-e8b2-4c8d-bec0-ecaeaf23a05a admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:47.255439 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ac76b-e8b2-4c8d-bec0-ecaeaf23a05a admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:47.256435 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ac76b-e8b2-4c8d-bec0-ecaeaf23a05a admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:47.257354 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348ac76b-e8b2-4c8d-bec0-ecaeaf23a05a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:47.263536 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348ac76b-e8b2-4c8d-bec0-ecaeaf23a05a admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:47.420465 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e0c553-57f1-4a62-89ea-fd1758c4f711 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:47.420703 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 831/1659] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:20:47 2022] POST /identity/v3/projects => generated 381 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:20:47.437374 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e0c553-57f1-4a62-89ea-fd1758c4f711 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:47.642401 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f19fb7-c57d-4a1a-8bdf-8fc32e33b92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:47.644618 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f19fb7-c57d-4a1a-8bdf-8fc32e33b92e admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:47.645781 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f19fb7-c57d-4a1a-8bdf-8fc32e33b92e admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:47.646335 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f19fb7-c57d-4a1a-8bdf-8fc32e33b92e admin admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:47.647929 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f19fb7-c57d-4a1a-8bdf-8fc32e33b92e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:47.651078 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f19fb7-c57d-4a1a-8bdf-8fc32e33b92e admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:47.702473 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348ac76b-e8b2-4c8d-bec0-ecaeaf23a05a admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:47.704491 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-348ac76b-e8b2-4c8d-bec0-ecaeaf23a05a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:47.748053 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348ac76b-e8b2-4c8d-bec0-ecaeaf23a05a admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:47.776480 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810c1940-1f8c-4f57-8d76-38beaaa9326f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:47.777074 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810c1940-1f8c-4f57-8d76-38beaaa9326f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:47.777566 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810c1940-1f8c-4f57-8d76-38beaaa9326f admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:47.777801 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 829/1660] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:20:47 2022] POST /identity/v3/users => generated 297 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:20:47.778320 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810c1940-1f8c-4f57-8d76-38beaaa9326f admin admin] PATH_INFO: `/v3/projects/b2c95d85cb07426ab0bf3f1d8dfd27c8` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:47.779265 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f19fb7-c57d-4a1a-8bdf-8fc32e33b92e admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:47.782732 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810c1940-1f8c-4f57-8d76-38beaaa9326f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2c95d85cb07426ab0bf3f1d8dfd27c8)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:47.791548 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810c1940-1f8c-4f57-8d76-38beaaa9326f admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:47.794569 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f19fb7-c57d-4a1a-8bdf-8fc32e33b92e admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:47.817503 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39fbed4-2f53-48e3-8074-03c989211baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:47.817744 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39fbed4-2f53-48e3-8074-03c989211baa admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:47.818013 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39fbed4-2f53-48e3-8074-03c989211baa admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:47.818882 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39fbed4-2f53-48e3-8074-03c989211baa admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:47.819984 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39fbed4-2f53-48e3-8074-03c989211baa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:47.822435 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39fbed4-2f53-48e3-8074-03c989211baa admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:47.838366 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 830/1661] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:20:47 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:20:47.845679 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3934b3-1c50-4a37-a420-dd3a38a18536 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:47.845977 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3934b3-1c50-4a37-a420-dd3a38a18536 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:47.846387 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3934b3-1c50-4a37-a420-dd3a38a18536 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:47.852407 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 831/1662] 10.222.0.33 () {54 vars in 960 bytes} [Thu Jun 2 19:20:47 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:20:47.854453 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3934b3-1c50-4a37-a420-dd3a38a18536 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:47.872960 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3934b3-1c50-4a37-a420-dd3a38a18536 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:47.888168 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d70e3c6-b76c-413c-bf45-827c885a586e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:47.888899 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d70e3c6-b76c-413c-bf45-827c885a586e admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:47.889331 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d70e3c6-b76c-413c-bf45-827c885a586e admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:47.889684 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d70e3c6-b76c-413c-bf45-827c885a586e admin admin] PATH_INFO: `/v3/projects/801a3fb14c9f49fda0e305d00253bb95/users/21cb2a1781e9442d9078070fea1185aa/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:47.912839 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d70e3c6-b76c-413c-bf45-827c885a586e admin admin] RBAC: Authorizing `identity:create_grant(project_id=801a3fb14c9f49fda0e305d00253bb95, user_id=21cb2a1781e9442d9078070fea1185aa, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:47.915831 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d70e3c6-b76c-413c-bf45-827c885a586e admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:47.945260 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:20:48.007784 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 832/1663] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:20:47 2022] PUT /identity/v3/projects/801a3fb14c9f49fda0e305d00253bb95/users/21cb2a1781e9442d9078070fea1185aa/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:20:48.010680 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38cbf4e-0f92-42ef-9d72-7cdf4c1802e8 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:48.010680 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38cbf4e-0f92-42ef-9d72-7cdf4c1802e8 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:48.010680 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38cbf4e-0f92-42ef-9d72-7cdf4c1802e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:48.028212 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-810c1940-1f8c-4f57-8d76-38beaaa9326f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2c95d85cb07426ab0bf3f1d8dfd27c8', 'request_id': 'req-810c1940-1f8c-4f57-8d76-38beaaa9326f'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:20:48.060978 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-e38cbf4e-0f92-42ef-9d72-7cdf4c1802e8 None None] MFA Rules not processed for user `73ae837e99764bebbe7a6e06ac9049e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:20:48.062571 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-e38cbf4e-0f92-42ef-9d72-7cdf4c1802e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:48.101229 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-810c1940-1f8c-4f57-8d76-38beaaa9326f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2c95d85cb07426ab0bf3f1d8dfd27c8', 'request_id': 'req-810c1940-1f8c-4f57-8d76-38beaaa9326f'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:20:48.121633 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-810c1940-1f8c-4f57-8d76-38beaaa9326f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2c95d85cb07426ab0bf3f1d8dfd27c8', 'request_id': 'req-810c1940-1f8c-4f57-8d76-38beaaa9326f'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:20:48.143406 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-810c1940-1f8c-4f57-8d76-38beaaa9326f admin admin] The token cache is being invalidate because project b2c95d85cb07426ab0bf3f1d8dfd27c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:20:48.144221 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-810c1940-1f8c-4f57-8d76-38beaaa9326f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:20:48.144762 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-810c1940-1f8c-4f57-8d76-38beaaa9326f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:20:48.150154 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 832/1664] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:20:47 2022] DELETE /identity/v3/projects/b2c95d85cb07426ab0bf3f1d8dfd27c8 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:20:48.150154 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810c1940-1f8c-4f57-8d76-38beaaa9326f admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:48.150375 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-810c1940-1f8c-4f57-8d76-38beaaa9326f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:48.170050 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 833/1665] 10.222.0.33 () {58 vars in 1057 bytes} [Thu Jun 2 19:20:48 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:20:48.170876 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38cbf4e-0f92-42ef-9d72-7cdf4c1802e8 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:48.173107 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-e38cbf4e-0f92-42ef-9d72-7cdf4c1802e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:48.192566 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:20:48.258930 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810c1940-1f8c-4f57-8d76-38beaaa9326f admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:48.266006 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0869373-9163-43f5-a523-231f11553e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:48.267023 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0869373-9163-43f5-a523-231f11553e03 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:48.268067 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0869373-9163-43f5-a523-231f11553e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:48.268531 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0869373-9163-43f5-a523-231f11553e03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:48.269454 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0869373-9163-43f5-a523-231f11553e03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:48.271568 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0869373-9163-43f5-a523-231f11553e03 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:48.271831 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38cbf4e-0f92-42ef-9d72-7cdf4c1802e8 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:48.278276 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd80174-1289-4b64-b598-d735fea07820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:48.279245 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd80174-1289-4b64-b598-d735fea07820 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:48.279870 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd80174-1289-4b64-b598-d735fea07820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:48.280168 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd80174-1289-4b64-b598-d735fea07820 admin admin] PATH_INFO: `/v3/users/3fc2f28b4a6e4e1b9fbbbe111eba5b4e` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:48.281769 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd80174-1289-4b64-b598-d735fea07820 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fc2f28b4a6e4e1b9fbbbe111eba5b4e)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:48.286682 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd80174-1289-4b64-b598-d735fea07820 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:48.293007 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 833/1666] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:20:48 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:20:48.293007 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0869373-9163-43f5-a523-231f11553e03 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:48.318203 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0869373-9163-43f5-a523-231f11553e03 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:48.328161 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bd275c-8f6a-442b-bc40-efe14ffd5a3b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73ae837e99764bebbe7a6e06ac9049e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:48.329327 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd275c-8f6a-442b-bc40-efe14ffd5a3b service placement] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:48.330440 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd275c-8f6a-442b-bc40-efe14ffd5a3b service placement] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:48.330999 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd275c-8f6a-442b-bc40-efe14ffd5a3b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:48.333374 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-56bd275c-8f6a-442b-bc40-efe14ffd5a3b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:48.352202 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bd275c-8f6a-442b-bc40-efe14ffd5a3b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:48.356253 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bd275c-8f6a-442b-bc40-efe14ffd5a3b service placement] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:48.374309 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 834/1667] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:20:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:20:48.375510 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bd275c-8f6a-442b-bc40-efe14ffd5a3b service placement] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:48.388715 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bd275c-8f6a-442b-bc40-efe14ffd5a3b service placement] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:48.396116 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bca40c-d928-49fa-b4f2-0019c406f8d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:48.397852 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bca40c-d928-49fa-b4f2-0019c406f8d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:48.398358 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bca40c-d928-49fa-b4f2-0019c406f8d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:48.399113 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bca40c-d928-49fa-b4f2-0019c406f8d4 admin admin] PATH_INFO: `/v3/projects/801a3fb14c9f49fda0e305d00253bb95/users/21cb2a1781e9442d9078070fea1185aa/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:48.403880 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bca40c-d928-49fa-b4f2-0019c406f8d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=801a3fb14c9f49fda0e305d00253bb95, user_id=21cb2a1781e9442d9078070fea1185aa, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:48.407593 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bca40c-d928-49fa-b4f2-0019c406f8d4 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:48.434802 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 835/1668] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:20:48 2022] PUT /identity/v3/projects/801a3fb14c9f49fda0e305d00253bb95/users/21cb2a1781e9442d9078070fea1185aa/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:20:48.443655 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99c2eb0-c05e-424a-a4d1-5870e58c869a None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:48.444104 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99c2eb0-c05e-424a-a4d1-5870e58c869a None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:48.444627 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99c2eb0-c05e-424a-a4d1-5870e58c869a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:48.513018 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-c99c2eb0-c05e-424a-a4d1-5870e58c869a None None] MFA Rules not processed for user `21cb2a1781e9442d9078070fea1185aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:20:48.514636 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-c99c2eb0-c05e-424a-a4d1-5870e58c869a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:48.581187 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-0cd80174-1289-4b64-b598-d735fea07820 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fc2f28b4a6e4e1b9fbbbe111eba5b4e'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:20:48.601432 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-0cd80174-1289-4b64-b598-d735fea07820 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fc2f28b4a6e4e1b9fbbbe111eba5b4e'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:20:48.602191 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-0cd80174-1289-4b64-b598-d735fea07820 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fc2f28b4a6e4e1b9fbbbe111eba5b4e'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:20:48.658609 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 836/1669] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:20:48 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 19:20:48.660449 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99c2eb0-c05e-424a-a4d1-5870e58c869a None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:48.667863 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-c99c2eb0-c05e-424a-a4d1-5870e58c869a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:48.691045 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-0cd80174-1289-4b64-b598-d735fea07820 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fc2f28b4a6e4e1b9fbbbe111eba5b4e'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:20:48.692694 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-0cd80174-1289-4b64-b598-d735fea07820 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fc2f28b4a6e4e1b9fbbbe111eba5b4e'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:20:48.749325 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 834/1670] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:20:48 2022] DELETE /identity/v3/users/3fc2f28b4a6e4e1b9fbbbe111eba5b4e => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:20:48.754026 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd80174-1289-4b64-b598-d735fea07820 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:48.757656 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-0cd80174-1289-4b64-b598-d735fea07820 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:48.875139 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99c2eb0-c05e-424a-a4d1-5870e58c869a None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:49.004942 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd80174-1289-4b64-b598-d735fea07820 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:49.014612 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68051e93-9cdd-460b-9e7c-55d1c83b608d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:49.015397 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68051e93-9cdd-460b-9e7c-55d1c83b608d service cinder] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:49.015693 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68051e93-9cdd-460b-9e7c-55d1c83b608d service cinder] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:49.015972 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68051e93-9cdd-460b-9e7c-55d1c83b608d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:49.020314 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87975a5e-dc65-4d3a-9cdd-eca3af3bc03e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:49.021603 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87975a5e-dc65-4d3a-9cdd-eca3af3bc03e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:49.022051 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87975a5e-dc65-4d3a-9cdd-eca3af3bc03e admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:49.022464 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87975a5e-dc65-4d3a-9cdd-eca3af3bc03e admin admin] PATH_INFO: `/v3/users/0424095acdf048cc8ebaef0499abff9b` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:49.025278 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87975a5e-dc65-4d3a-9cdd-eca3af3bc03e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0424095acdf048cc8ebaef0499abff9b)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:49.027865 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87975a5e-dc65-4d3a-9cdd-eca3af3bc03e admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:49.245781 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68051e93-9cdd-460b-9e7c-55d1c83b608d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:49.248797 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68051e93-9cdd-460b-9e7c-55d1c83b608d service cinder] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:49.275028 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 835/1671] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:20:48 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:20:49.310246 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-87975a5e-dc65-4d3a-9cdd-eca3af3bc03e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0424095acdf048cc8ebaef0499abff9b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:20:49.325525 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-87975a5e-dc65-4d3a-9cdd-eca3af3bc03e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0424095acdf048cc8ebaef0499abff9b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:20:49.394531 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-87975a5e-dc65-4d3a-9cdd-eca3af3bc03e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0424095acdf048cc8ebaef0499abff9b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:20:49.395536 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-87975a5e-dc65-4d3a-9cdd-eca3af3bc03e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0424095acdf048cc8ebaef0499abff9b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:20:49.396231 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-87975a5e-dc65-4d3a-9cdd-eca3af3bc03e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0424095acdf048cc8ebaef0499abff9b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:20:49.427716 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 837/1672] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:20:48 2022] DELETE /identity/v3/users/0424095acdf048cc8ebaef0499abff9b => generated 0 bytes in 769 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:20:49.433282 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5b00d5-a86d-4157-98da-8d6801cf0a00 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:49.433559 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5b00d5-a86d-4157-98da-8d6801cf0a00 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:49.433853 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5b00d5-a86d-4157-98da-8d6801cf0a00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:49.478997 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:20:49.496612 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-aa5b00d5-a86d-4157-98da-8d6801cf0a00 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:20:49.498721 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-aa5b00d5-a86d-4157-98da-8d6801cf0a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:49.585901 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87975a5e-dc65-4d3a-9cdd-eca3af3bc03e admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:49.585901 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-87975a5e-dc65-4d3a-9cdd-eca3af3bc03e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:49.702460 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:20:49.720685 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 836/1673] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:20:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:20:49.721384 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5b00d5-a86d-4157-98da-8d6801cf0a00 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:49.723217 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-aa5b00d5-a86d-4157-98da-8d6801cf0a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:49.759921 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87975a5e-dc65-4d3a-9cdd-eca3af3bc03e admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:49.766447 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba90b550-586e-418a-b636-d7ee2cc2187a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:49.768172 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba90b550-586e-418a-b636-d7ee2cc2187a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:49.768470 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba90b550-586e-418a-b636-d7ee2cc2187a admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:49.770589 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba90b550-586e-418a-b636-d7ee2cc2187a admin admin] PATH_INFO: `/v3/projects/2ec4f05ed28b4a89839c291b729d7920` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:49.770589 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba90b550-586e-418a-b636-d7ee2cc2187a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ec4f05ed28b4a89839c291b729d7920)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:49.772552 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba90b550-586e-418a-b636-d7ee2cc2187a admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:49.870984 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5b00d5-a86d-4157-98da-8d6801cf0a00 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:49.879367 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd01afcc-ed4e-4fb6-9a66-de7350ad099d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:49.880282 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd01afcc-ed4e-4fb6-9a66-de7350ad099d service cinder] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:49.880636 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd01afcc-ed4e-4fb6-9a66-de7350ad099d service cinder] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:49.880987 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd01afcc-ed4e-4fb6-9a66-de7350ad099d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:49.888891 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-dd01afcc-ed4e-4fb6-9a66-de7350ad099d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:49.933528 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-ba90b550-586e-418a-b636-d7ee2cc2187a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ec4f05ed28b4a89839c291b729d7920', 'request_id': 'req-ba90b550-586e-418a-b636-d7ee2cc2187a'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:20:50.026289 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd01afcc-ed4e-4fb6-9a66-de7350ad099d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:50.027919 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd01afcc-ed4e-4fb6-9a66-de7350ad099d service cinder] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:50.045621 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 837/1674] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:20:49 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:20:50.046574 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd01afcc-ed4e-4fb6-9a66-de7350ad099d service cinder] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:50.048043 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-dd01afcc-ed4e-4fb6-9a66-de7350ad099d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:50.073016 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-ba90b550-586e-418a-b636-d7ee2cc2187a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ec4f05ed28b4a89839c291b729d7920', 'request_id': 'req-ba90b550-586e-418a-b636-d7ee2cc2187a'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:20:50.080264 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-ba90b550-586e-418a-b636-d7ee2cc2187a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ec4f05ed28b4a89839c291b729d7920', 'request_id': 'req-ba90b550-586e-418a-b636-d7ee2cc2187a'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:20:50.137512 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-ba90b550-586e-418a-b636-d7ee2cc2187a admin admin] The token cache is being invalidate because project 2ec4f05ed28b4a89839c291b729d7920 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:20:50.137825 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-ba90b550-586e-418a-b636-d7ee2cc2187a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:20:50.143319 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-ba90b550-586e-418a-b636-d7ee2cc2187a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:20:50.155065 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 838/1675] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:20:49 2022] DELETE /identity/v3/projects/2ec4f05ed28b4a89839c291b729d7920 => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:20:50.156575 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d769692e-64ad-40a9-a0fb-50b93c2abb9e None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:50.156874 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d769692e-64ad-40a9-a0fb-50b93c2abb9e None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:50.157165 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d769692e-64ad-40a9-a0fb-50b93c2abb9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:50.206874 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-d769692e-64ad-40a9-a0fb-50b93c2abb9e None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:20:50.208662 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-d769692e-64ad-40a9-a0fb-50b93c2abb9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:50.217896 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd01afcc-ed4e-4fb6-9a66-de7350ad099d service cinder] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:50.352401 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4dc488-d88c-4233-8c15-625b89505b34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:50.353086 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4dc488-d88c-4233-8c15-625b89505b34 service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:50.353371 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4dc488-d88c-4233-8c15-625b89505b34 service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:50.353647 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4dc488-d88c-4233-8c15-625b89505b34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:50.356406 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-db4dc488-d88c-4233-8c15-625b89505b34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:50.418881 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 839/1676] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:20:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:20:50.421159 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbad7f1-94e3-4d76-8065-6d47e7700700 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:50.421460 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbad7f1-94e3-4d76-8065-6d47e7700700 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:50.421746 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbad7f1-94e3-4d76-8065-6d47e7700700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:50.451818 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-bfbad7f1-94e3-4d76-8065-6d47e7700700 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:20:50.454459 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-bfbad7f1-94e3-4d76-8065-6d47e7700700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:50.471496 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 840/1677] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:20:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:20:50.471868 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbad7f1-94e3-4d76-8065-6d47e7700700 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:50.484431 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbad7f1-94e3-4d76-8065-6d47e7700700 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:50.492875 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e06a2b-a9f1-410e-9078-25e8fea4e7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:50.493624 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e06a2b-a9f1-410e-9078-25e8fea4e7b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:50.493918 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e06a2b-a9f1-410e-9078-25e8fea4e7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:50.494199 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e06a2b-a9f1-410e-9078-25e8fea4e7b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:50.495099 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e06a2b-a9f1-410e-9078-25e8fea4e7b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:20:50.495417 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e06a2b-a9f1-410e-9078-25e8fea4e7b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:50.497653 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e06a2b-a9f1-410e-9078-25e8fea4e7b7 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:50.517874 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 841/1678] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 2 19:20:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:20:50.526133 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e06a2b-a9f1-410e-9078-25e8fea4e7b7 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:50.542943 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e06a2b-a9f1-410e-9078-25e8fea4e7b7 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:50.549745 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecc0f03-68c2-4252-9d68-0676d709d100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:50.550497 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc0f03-68c2-4252-9d68-0676d709d100 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:50.550905 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc0f03-68c2-4252-9d68-0676d709d100 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:50.551194 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc0f03-68c2-4252-9d68-0676d709d100 admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:50.551985 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecc0f03-68c2-4252-9d68-0676d709d100 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:50.555062 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecc0f03-68c2-4252-9d68-0676d709d100 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:50.661784 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4dc488-d88c-4233-8c15-625b89505b34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:50.663925 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4dc488-d88c-4233-8c15-625b89505b34 service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:50.693294 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 838/1679] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:20:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 647 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:20:50.713365 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 842/1680] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:20:50 2022] POST /identity/v3/projects => generated 383 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:20:50.717921 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4dc488-d88c-4233-8c15-625b89505b34 service neutron] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:50.736061 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4dc488-d88c-4233-8c15-625b89505b34 service neutron] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:50.824088 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4604818-e06a-45e1-ba63-a546231ec983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:50.825153 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4604818-e06a-45e1-ba63-a546231ec983 admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:50.825596 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4604818-e06a-45e1-ba63-a546231ec983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:50.827013 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4604818-e06a-45e1-ba63-a546231ec983 admin admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:50.827013 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4604818-e06a-45e1-ba63-a546231ec983 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:50.829561 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4604818-e06a-45e1-ba63-a546231ec983 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:50.942693 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 839/1681] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:20:50 2022] POST /identity/v3/users => generated 292 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:20:50.947400 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecc0f03-68c2-4252-9d68-0676d709d100 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:50.966715 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecc0f03-68c2-4252-9d68-0676d709d100 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:50.983807 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd6d992-b0ae-4771-819c-1108d60f6ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:50.984504 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd6d992-b0ae-4771-819c-1108d60f6ecc admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:50.984797 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd6d992-b0ae-4771-819c-1108d60f6ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:50.985643 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd6d992-b0ae-4771-819c-1108d60f6ecc admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:50.986527 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd6d992-b0ae-4771-819c-1108d60f6ecc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:50.988251 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd6d992-b0ae-4771-819c-1108d60f6ecc admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:51.019040 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 843/1682] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:20:50 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:20:51.019265 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4604818-e06a-45e1-ba63-a546231ec983 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:51.053058 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4604818-e06a-45e1-ba63-a546231ec983 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:51.083451 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9499b3cd-e01c-4e5b-9e0a-ddc3397ab900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:51.084045 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9499b3cd-e01c-4e5b-9e0a-ddc3397ab900 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:51.084330 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9499b3cd-e01c-4e5b-9e0a-ddc3397ab900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:51.084610 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9499b3cd-e01c-4e5b-9e0a-ddc3397ab900 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:51.085359 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9499b3cd-e01c-4e5b-9e0a-ddc3397ab900 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:51.087066 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9499b3cd-e01c-4e5b-9e0a-ddc3397ab900 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:51.117261 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 840/1683] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:20:51 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:20:51.122457 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd6d992-b0ae-4771-819c-1108d60f6ecc admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:51.151578 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd6d992-b0ae-4771-819c-1108d60f6ecc admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:51.157130 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aecfb10-15ae-4369-ae02-2266df2c1e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:51.158089 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aecfb10-15ae-4369-ae02-2266df2c1e73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:51.158381 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aecfb10-15ae-4369-ae02-2266df2c1e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:51.158662 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aecfb10-15ae-4369-ae02-2266df2c1e73 admin admin] PATH_INFO: `/v3/projects/8ffb28cd3ad84f31947362c2748336eb/users/5ee08b16f3d74d69b52e7dd18902ac16/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:51.196791 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aecfb10-15ae-4369-ae02-2266df2c1e73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ffb28cd3ad84f31947362c2748336eb, user_id=5ee08b16f3d74d69b52e7dd18902ac16, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:51.199536 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aecfb10-15ae-4369-ae02-2266df2c1e73 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:51.313949 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 844/1684] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:20:51 2022] PUT /identity/v3/projects/8ffb28cd3ad84f31947362c2748336eb/users/5ee08b16f3d74d69b52e7dd18902ac16/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:20:51.317791 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36855edd-ae59-4ac4-b6e4-e3062873f860 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:51.317791 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36855edd-ae59-4ac4-b6e4-e3062873f860 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:51.317791 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36855edd-ae59-4ac4-b6e4-e3062873f860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:51.370396 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-36855edd-ae59-4ac4-b6e4-e3062873f860 None None] MFA Rules not processed for user `5ee08b16f3d74d69b52e7dd18902ac16`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:20:51.376329 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-36855edd-ae59-4ac4-b6e4-e3062873f860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:51.388630 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:20:51.447301 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aecfb10-15ae-4369-ae02-2266df2c1e73 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:51.449412 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-9aecfb10-15ae-4369-ae02-2266df2c1e73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:51.532719 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:20:51.559554 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 841/1685] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:20:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:20:51.560958 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36855edd-ae59-4ac4-b6e4-e3062873f860 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:51.638357 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aecfb10-15ae-4369-ae02-2266df2c1e73 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:51.648310 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efcdfc0-265c-4471-9e43-8272892495ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:51.649215 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efcdfc0-265c-4471-9e43-8272892495ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:51.649497 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efcdfc0-265c-4471-9e43-8272892495ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:51.649937 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efcdfc0-265c-4471-9e43-8272892495ed admin admin] PATH_INFO: `/v3/projects/2490a4964da74cf1837be4c8843ad906` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:51.651484 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efcdfc0-265c-4471-9e43-8272892495ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=2490a4964da74cf1837be4c8843ad906)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:51.653685 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efcdfc0-265c-4471-9e43-8272892495ed admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:51.696386 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36855edd-ae59-4ac4-b6e4-e3062873f860 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:51.707896 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:20:51.711242 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb0a7b5-69e1-450a-9fe2-73174705f362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:51.711915 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0a7b5-69e1-450a-9fe2-73174705f362 admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:51.712194 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0a7b5-69e1-450a-9fe2-73174705f362 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:51.712468 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0a7b5-69e1-450a-9fe2-73174705f362 admin admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:51.713216 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb0a7b5-69e1-450a-9fe2-73174705f362 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:51.715670 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb0a7b5-69e1-450a-9fe2-73174705f362 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:51.779849 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-3efcdfc0-265c-4471-9e43-8272892495ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2490a4964da74cf1837be4c8843ad906', 'request_id': 'req-3efcdfc0-265c-4471-9e43-8272892495ed'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:20:51.808133 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 842/1686] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:20:51 2022] POST /identity/v3/projects => generated 383 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:20:51.809485 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb0a7b5-69e1-450a-9fe2-73174705f362 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:51.872947 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-3efcdfc0-265c-4471-9e43-8272892495ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2490a4964da74cf1837be4c8843ad906', 'request_id': 'req-3efcdfc0-265c-4471-9e43-8272892495ed'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:20:51.881684 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-3efcdfc0-265c-4471-9e43-8272892495ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2490a4964da74cf1837be4c8843ad906', 'request_id': 'req-3efcdfc0-265c-4471-9e43-8272892495ed'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:20:51.915434 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-3efcdfc0-265c-4471-9e43-8272892495ed admin admin] The token cache is being invalidate because project 2490a4964da74cf1837be4c8843ad906 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:20:51.916176 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-3efcdfc0-265c-4471-9e43-8272892495ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:20:51.935935 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-3efcdfc0-265c-4471-9e43-8272892495ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:20:51.942354 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 845/1687] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:20:51 2022] DELETE /identity/v3/projects/2490a4964da74cf1837be4c8843ad906 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:20:51.964438 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1cf5fb-41fa-4821-9fe5-5fcfdb604679 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:51.965060 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1cf5fb-41fa-4821-9fe5-5fcfdb604679 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:51.965288 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1cf5fb-41fa-4821-9fe5-5fcfdb604679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:52.000147 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb0a7b5-69e1-450a-9fe2-73174705f362 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:52.011921 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01780653-2a19-40d9-be3c-7ff56e512aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:52.012620 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01780653-2a19-40d9-be3c-7ff56e512aa5 admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:52.015129 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01780653-2a19-40d9-be3c-7ff56e512aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:52.016410 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01780653-2a19-40d9-be3c-7ff56e512aa5 admin admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:52.017381 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-7b1cf5fb-41fa-4821-9fe5-5fcfdb604679 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:20:52.017938 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01780653-2a19-40d9-be3c-7ff56e512aa5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:52.019041 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-7b1cf5fb-41fa-4821-9fe5-5fcfdb604679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:52.023800 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01780653-2a19-40d9-be3c-7ff56e512aa5 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:52.040739 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 846/1688] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:20:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:20:52.049398 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5625f351-29d3-48a5-9b58-806146a65aa5 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:52.055319 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5625f351-29d3-48a5-9b58-806146a65aa5 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:52.055882 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5625f351-29d3-48a5-9b58-806146a65aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:52.089909 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-5625f351-29d3-48a5-9b58-806146a65aa5 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:20:52.091961 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-5625f351-29d3-48a5-9b58-806146a65aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:52.107597 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 847/1689] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:20:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:20:52.109892 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5625f351-29d3-48a5-9b58-806146a65aa5 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:52.127379 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5625f351-29d3-48a5-9b58-806146a65aa5 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:52.143067 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 843/1690] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:20:51 2022] POST /identity/v3/users => generated 298 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:20:52.143874 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01780653-2a19-40d9-be3c-7ff56e512aa5 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:52.147043 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-01780653-2a19-40d9-be3c-7ff56e512aa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:52.161909 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5662fdb6-d751-4a7f-8b28-bc3c335b0ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:52.162954 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5662fdb6-d751-4a7f-8b28-bc3c335b0ce1 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:52.165051 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5662fdb6-d751-4a7f-8b28-bc3c335b0ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:52.166063 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5662fdb6-d751-4a7f-8b28-bc3c335b0ce1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:52.168797 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5662fdb6-d751-4a7f-8b28-bc3c335b0ce1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:20:52.169144 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5662fdb6-d751-4a7f-8b28-bc3c335b0ce1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:52.171580 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5662fdb6-d751-4a7f-8b28-bc3c335b0ce1 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:52.177735 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01780653-2a19-40d9-be3c-7ff56e512aa5 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:52.185469 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc93a8e-6b96-4335-93fb-a7718812efe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:52.186467 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc93a8e-6b96-4335-93fb-a7718812efe4 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:52.187181 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc93a8e-6b96-4335-93fb-a7718812efe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:52.187643 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc93a8e-6b96-4335-93fb-a7718812efe4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:52.188542 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc93a8e-6b96-4335-93fb-a7718812efe4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:52.190506 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc93a8e-6b96-4335-93fb-a7718812efe4 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:52.190722 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 848/1691] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 2 19:20:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:20:52.194666 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5662fdb6-d751-4a7f-8b28-bc3c335b0ce1 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:52.202963 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 844/1692] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:20:52 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:20:52.207814 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc93a8e-6b96-4335-93fb-a7718812efe4 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:52.210883 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5662fdb6-d751-4a7f-8b28-bc3c335b0ce1 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:52.218645 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28200571-8187-4205-b9bb-9a1259396728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:52.219334 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc93a8e-6b96-4335-93fb-a7718812efe4 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:52.219853 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28200571-8187-4205-b9bb-9a1259396728 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:52.221368 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28200571-8187-4205-b9bb-9a1259396728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:52.221869 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28200571-8187-4205-b9bb-9a1259396728 admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:52.222843 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28200571-8187-4205-b9bb-9a1259396728 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:52.225286 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28200571-8187-4205-b9bb-9a1259396728 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:52.226970 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270dc75c-68cc-4b0d-a753-e56576032e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:52.228509 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270dc75c-68cc-4b0d-a753-e56576032e6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:52.229012 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270dc75c-68cc-4b0d-a753-e56576032e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:52.229433 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270dc75c-68cc-4b0d-a753-e56576032e6f admin admin] PATH_INFO: `/v3/projects/11cb34b81f8b4892a79e034de933c4ea/users/cf952a948ff344e1ac373c66a48b0ca8/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:52.253633 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270dc75c-68cc-4b0d-a753-e56576032e6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=11cb34b81f8b4892a79e034de933c4ea, user_id=cf952a948ff344e1ac373c66a48b0ca8, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:52.257696 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270dc75c-68cc-4b0d-a753-e56576032e6f admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:52.349432 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 845/1693] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:20:52 2022] PUT /identity/v3/projects/11cb34b81f8b4892a79e034de933c4ea/users/cf952a948ff344e1ac373c66a48b0ca8/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:20:52.353674 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270dc75c-68cc-4b0d-a753-e56576032e6f admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:52.435766 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 849/1694] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:20:52 2022] POST /identity/v3/projects => generated 365 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:20:52.437407 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28200571-8187-4205-b9bb-9a1259396728 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:52.542551 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270dc75c-68cc-4b0d-a753-e56576032e6f admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:52.612442 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28200571-8187-4205-b9bb-9a1259396728 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:52.620610 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdc90a6-4b6f-4ab2-a077-a47d92f5f283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:52.622150 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc90a6-4b6f-4ab2-a077-a47d92f5f283 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:52.622639 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc90a6-4b6f-4ab2-a077-a47d92f5f283 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:52.623932 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc90a6-4b6f-4ab2-a077-a47d92f5f283 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:52.648297 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdc90a6-4b6f-4ab2-a077-a47d92f5f283 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:52.655060 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdc90a6-4b6f-4ab2-a077-a47d92f5f283 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:52.711894 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:20:52.732923 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d68e591-83ae-4f33-849b-aaa4daf287a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:52.735390 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d68e591-83ae-4f33-849b-aaa4daf287a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:52.735905 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d68e591-83ae-4f33-849b-aaa4daf287a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:52.737948 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d68e591-83ae-4f33-849b-aaa4daf287a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:52.738967 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d68e591-83ae-4f33-849b-aaa4daf287a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:52.742245 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d68e591-83ae-4f33-849b-aaa4daf287a8 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:52.784218 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 846/1695] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:20:52 2022] GET /identity/v3/roles => generated 1420 bytes in 431 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 02 19:20:52.785495 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d68e591-83ae-4f33-849b-aaa4daf287a8 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:52.788357 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 850/1696] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:20:52 2022] POST /identity/v3/users => generated 283 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:20:52.792372 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdc90a6-4b6f-4ab2-a077-a47d92f5f283 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:52.813792 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d68e591-83ae-4f33-849b-aaa4daf287a8 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:52.823963 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de87cc62-51af-4b35-8604-1ea6d9e2831e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:52.825200 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de87cc62-51af-4b35-8604-1ea6d9e2831e admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:52.825729 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdc90a6-4b6f-4ab2-a077-a47d92f5f283 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:52.828024 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de87cc62-51af-4b35-8604-1ea6d9e2831e admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:52.828424 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de87cc62-51af-4b35-8604-1ea6d9e2831e admin admin] PATH_INFO: `/v3/projects/11cb34b81f8b4892a79e034de933c4ea/users/cf952a948ff344e1ac373c66a48b0ca8/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:52.835706 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de87cc62-51af-4b35-8604-1ea6d9e2831e admin admin] RBAC: Authorizing `identity:create_grant(project_id=11cb34b81f8b4892a79e034de933c4ea, user_id=cf952a948ff344e1ac373c66a48b0ca8, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:52.844682 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de87cc62-51af-4b35-8604-1ea6d9e2831e admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:52.848569 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28da3ab5-e302-4b84-8272-ff1c26fbb14e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:52.849338 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da3ab5-e302-4b84-8272-ff1c26fbb14e admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:52.849626 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da3ab5-e302-4b84-8272-ff1c26fbb14e admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:52.849908 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da3ab5-e302-4b84-8272-ff1c26fbb14e admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:52.850669 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28da3ab5-e302-4b84-8272-ff1c26fbb14e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:52.852343 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28da3ab5-e302-4b84-8272-ff1c26fbb14e admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:52.890868 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 847/1697] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:20:52 2022] PUT /identity/v3/projects/11cb34b81f8b4892a79e034de933c4ea/users/cf952a948ff344e1ac373c66a48b0ca8/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:20:52.895020 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e85942b-2e80-4bcc-84b1-69f66e497dcf None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:52.895707 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e85942b-2e80-4bcc-84b1-69f66e497dcf None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:52.896432 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e85942b-2e80-4bcc-84b1-69f66e497dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:52.914551 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 851/1698] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:20:52 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:20:52.917105 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28da3ab5-e302-4b84-8272-ff1c26fbb14e admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:52.955647 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28da3ab5-e302-4b84-8272-ff1c26fbb14e admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:52.965017 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:20:52.966723 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-3e85942b-2e80-4bcc-84b1-69f66e497dcf None None] MFA Rules not processed for user `cf952a948ff344e1ac373c66a48b0ca8`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:20:52.968224 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-3e85942b-2e80-4bcc-84b1-69f66e497dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:53.151647 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 848/1699] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:20:52 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:20:53.162073 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e85942b-2e80-4bcc-84b1-69f66e497dcf None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:53.169219 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-3e85942b-2e80-4bcc-84b1-69f66e497dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:53.178488 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23993a3e-2f80-48b3-9828-fdf07520545c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:53.179260 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23993a3e-2f80-48b3-9828-fdf07520545c admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:53.182128 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23993a3e-2f80-48b3-9828-fdf07520545c admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:53.185881 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23993a3e-2f80-48b3-9828-fdf07520545c admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:53.186798 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23993a3e-2f80-48b3-9828-fdf07520545c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:53.190338 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23993a3e-2f80-48b3-9828-fdf07520545c admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:53.223849 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 852/1700] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:20:52 2022] GET /identity/v3/roles => generated 1420 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:20:53.228563 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23993a3e-2f80-48b3-9828-fdf07520545c admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:53.251142 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23993a3e-2f80-48b3-9828-fdf07520545c admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:53.266406 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cd36a9-805b-4e84-8395-2b156cf8d5c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:53.267499 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd36a9-805b-4e84-8395-2b156cf8d5c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:53.267786 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd36a9-805b-4e84-8395-2b156cf8d5c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:53.268066 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd36a9-805b-4e84-8395-2b156cf8d5c4 admin admin] PATH_INFO: `/v3/projects/7a09abe3e85c4c83a6401029a539da4b/users/93eab725113c47e1aadbe00b41b86660/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:53.285076 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e85942b-2e80-4bcc-84b1-69f66e497dcf None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:53.292509 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e786b04f-44d1-49a8-a9a2-7f4d8f469986 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:53.299760 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e786b04f-44d1-49a8-a9a2-7f4d8f469986 service cinder] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:53.299760 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e786b04f-44d1-49a8-a9a2-7f4d8f469986 service cinder] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:53.300152 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e786b04f-44d1-49a8-a9a2-7f4d8f469986 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:53.315240 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cd36a9-805b-4e84-8395-2b156cf8d5c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a09abe3e85c4c83a6401029a539da4b, user_id=93eab725113c47e1aadbe00b41b86660, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:53.326774 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cd36a9-805b-4e84-8395-2b156cf8d5c4 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:53.327846 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e786b04f-44d1-49a8-a9a2-7f4d8f469986 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:53.329588 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e786b04f-44d1-49a8-a9a2-7f4d8f469986 service cinder] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:53.345341 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 849/1701] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:20:53 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 02 19:20:53.404142 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:20:53.459507 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 853/1702] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:20:53 2022] PUT /identity/v3/projects/7a09abe3e85c4c83a6401029a539da4b/users/93eab725113c47e1aadbe00b41b86660/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 02 19:20:53.465941 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d7a75-b93a-40c6-988f-ad597f946817 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:53.466306 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d7a75-b93a-40c6-988f-ad597f946817 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:53.466675 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d7a75-b93a-40c6-988f-ad597f946817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:53.516734 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-051d7a75-b93a-40c6-988f-ad597f946817 None None] MFA Rules not processed for user `93eab725113c47e1aadbe00b41b86660`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:20:53.520634 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-051d7a75-b93a-40c6-988f-ad597f946817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:53.597821 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cd36a9-805b-4e84-8395-2b156cf8d5c4 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:53.697315 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 850/1703] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:20:53 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:20:53.700709 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f473ea-c1a5-4354-8bbf-2972faf33013 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:53.702004 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f473ea-c1a5-4354-8bbf-2972faf33013 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:53.702870 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f473ea-c1a5-4354-8bbf-2972faf33013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:53.742819 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cd36a9-805b-4e84-8395-2b156cf8d5c4 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:53.752564 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eb62c1-a498-42c1-9719-cbaed10731d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:53.753346 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb62c1-a498-42c1-9719-cbaed10731d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:53.754900 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb62c1-a498-42c1-9719-cbaed10731d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:53.755458 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb62c1-a498-42c1-9719-cbaed10731d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:53.757887 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-25f473ea-c1a5-4354-8bbf-2972faf33013 None None] MFA Rules not processed for user `04f64813f09348eb91cb7bb2869f7612`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:20:53.760086 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-25f473ea-c1a5-4354-8bbf-2972faf33013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:53.762534 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-04eb62c1-a498-42c1-9719-cbaed10731d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:53.887057 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eb62c1-a498-42c1-9719-cbaed10731d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:53.889359 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eb62c1-a498-42c1-9719-cbaed10731d4 service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:53.902831 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 851/1704] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Jun 2 19:20:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:20:53.910641 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f473ea-c1a5-4354-8bbf-2972faf33013 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:53.940377 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 854/1705] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:20:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 02 19:20:54.068558 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f473ea-c1a5-4354-8bbf-2972faf33013 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:54.083063 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0072b412-54e7-46b6-b9b0-a87d9f075129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:54.083678 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0072b412-54e7-46b6-b9b0-a87d9f075129 admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:54.083962 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0072b412-54e7-46b6-b9b0-a87d9f075129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:54.084407 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0072b412-54e7-46b6-b9b0-a87d9f075129 admin admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:54.090163 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0072b412-54e7-46b6-b9b0-a87d9f075129 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:54.092473 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0072b412-54e7-46b6-b9b0-a87d9f075129 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:54.235696 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 852/1706] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:20:53 2022] POST /identity/v3/projects => generated 365 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:20:54.241511 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eb62c1-a498-42c1-9719-cbaed10731d4 service cinder] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:54.270713 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eb62c1-a498-42c1-9719-cbaed10731d4 service cinder] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:54.321744 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:20:54.372212 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0072b412-54e7-46b6-b9b0-a87d9f075129 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:54.374512 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-0072b412-54e7-46b6-b9b0-a87d9f075129 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:54.421871 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:20:54.445236 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7b9951-f996-4024-ad66-480eb4844e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:54.447241 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7b9951-f996-4024-ad66-480eb4844e84 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:54.447982 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7b9951-f996-4024-ad66-480eb4844e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:54.448420 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7b9951-f996-4024-ad66-480eb4844e84 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:54.449312 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7b9951-f996-4024-ad66-480eb4844e84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:54.451667 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7b9951-f996-4024-ad66-480eb4844e84 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:54.547252 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 855/1707] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:20:54 2022] POST /identity/v3/users => generated 289 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:20:54.553561 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7b9951-f996-4024-ad66-480eb4844e84 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:54.583001 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0072b412-54e7-46b6-b9b0-a87d9f075129 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:54.592121 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7b9951-f996-4024-ad66-480eb4844e84 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:54.596779 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ff8e2d-d115-420d-9c33-4244bce1fa5a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a59e1d67ff94186a3ebcfcb49938a87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:54.597453 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ff8e2d-d115-420d-9c33-4244bce1fa5a service swift] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:54.597739 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ff8e2d-d115-420d-9c33-4244bce1fa5a service swift] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:54.598032 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ff8e2d-d115-420d-9c33-4244bce1fa5a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:54.608485 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d0c64-7f5e-41b2-95d0-80791029feb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:54.609934 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d0c64-7f5e-41b2-95d0-80791029feb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:54.613869 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d0c64-7f5e-41b2-95d0-80791029feb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:54.614312 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d0c64-7f5e-41b2-95d0-80791029feb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:54.616621 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22d0c64-7f5e-41b2-95d0-80791029feb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:54.621742 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22d0c64-7f5e-41b2-95d0-80791029feb3 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:54.660883 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 856/1708] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:20:54 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:20:54.663551 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d0c64-7f5e-41b2-95d0-80791029feb3 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:54.686892 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d0c64-7f5e-41b2-95d0-80791029feb3 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:54.705766 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dd5bea-34d1-434d-bd0e-52981f1183cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:54.708235 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd5bea-34d1-434d-bd0e-52981f1183cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:54.708694 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd5bea-34d1-434d-bd0e-52981f1183cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:54.712166 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd5bea-34d1-434d-bd0e-52981f1183cd admin admin] PATH_INFO: `/v3/projects/c282e21215d74158aa1891734ac7b199/users/a3ac833d0e8c4202a0d60ec731d9899b/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:54.744744 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dd5bea-34d1-434d-bd0e-52981f1183cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=c282e21215d74158aa1891734ac7b199, user_id=a3ac833d0e8c4202a0d60ec731d9899b, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:54.747924 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dd5bea-34d1-434d-bd0e-52981f1183cd admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:54.762109 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ff8e2d-d115-420d-9c33-4244bce1fa5a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:54.763771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ff8e2d-d115-420d-9c33-4244bce1fa5a service swift] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:54.782154 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 853/1709] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 2 19:20:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 02 19:20:54.857694 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 857/1710] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:20:54 2022] PUT /identity/v3/projects/c282e21215d74158aa1891734ac7b199/users/a3ac833d0e8c4202a0d60ec731d9899b/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:20:54.863339 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ff8e2d-d115-420d-9c33-4244bce1fa5a service swift] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:54.897080 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ff8e2d-d115-420d-9c33-4244bce1fa5a service swift] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:55.161045 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63232e2-fbaa-4dc4-a049-2be4aae6a7b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:55.170146 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63232e2-fbaa-4dc4-a049-2be4aae6a7b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:55.170467 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63232e2-fbaa-4dc4-a049-2be4aae6a7b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:55.170795 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63232e2-fbaa-4dc4-a049-2be4aae6a7b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:55.172850 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63232e2-fbaa-4dc4-a049-2be4aae6a7b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:55.179157 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dd5bea-34d1-434d-bd0e-52981f1183cd admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:55.179449 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63232e2-fbaa-4dc4-a049-2be4aae6a7b8 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:55.202212 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dd5bea-34d1-434d-bd0e-52981f1183cd admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:55.221899 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e35e76-d08d-4cd9-ae2c-36aa61ca65a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:55.225153 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e35e76-d08d-4cd9-ae2c-36aa61ca65a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:55.226479 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e35e76-d08d-4cd9-ae2c-36aa61ca65a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:55.238266 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e35e76-d08d-4cd9-ae2c-36aa61ca65a5 admin admin] PATH_INFO: `/v3/users/5ee08b16f3d74d69b52e7dd18902ac16` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:55.242704 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 854/1711] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:20:54 2022] GET /identity/v3/roles => generated 1420 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:20:55.245217 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63232e2-fbaa-4dc4-a049-2be4aae6a7b8 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:55.245896 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e35e76-d08d-4cd9-ae2c-36aa61ca65a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ee08b16f3d74d69b52e7dd18902ac16)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:55.254266 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e35e76-d08d-4cd9-ae2c-36aa61ca65a5 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:55.274079 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:20:55.282288 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63232e2-fbaa-4dc4-a049-2be4aae6a7b8 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:55.311797 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702ba2f0-7bce-47a1-9f44-9ad7c669ef5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:55.314026 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ba2f0-7bce-47a1-9f44-9ad7c669ef5e admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:55.322802 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ba2f0-7bce-47a1-9f44-9ad7c669ef5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:55.323859 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ba2f0-7bce-47a1-9f44-9ad7c669ef5e admin admin] PATH_INFO: `/v3/projects/c282e21215d74158aa1891734ac7b199/users/a3ac833d0e8c4202a0d60ec731d9899b/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:55.329534 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702ba2f0-7bce-47a1-9f44-9ad7c669ef5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c282e21215d74158aa1891734ac7b199, user_id=a3ac833d0e8c4202a0d60ec731d9899b, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:55.340717 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702ba2f0-7bce-47a1-9f44-9ad7c669ef5e admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:55.378389 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 855/1712] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:20:55 2022] PUT /identity/v3/projects/c282e21215d74158aa1891734ac7b199/users/a3ac833d0e8c4202a0d60ec731d9899b/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:20:55.393445 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47a199c-27e7-47a2-9067-148888b00adb None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:55.393757 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47a199c-27e7-47a2-9067-148888b00adb None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:55.394091 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47a199c-27e7-47a2-9067-148888b00adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:55.482160 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-b47a199c-27e7-47a2-9067-148888b00adb None None] MFA Rules not processed for user `a3ac833d0e8c4202a0d60ec731d9899b`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:20:55.483815 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-b47a199c-27e7-47a2-9067-148888b00adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:55.595581 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:20:55.596868 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-08e35e76-d08d-4cd9-ae2c-36aa61ca65a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ee08b16f3d74d69b52e7dd18902ac16'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:20:55.618362 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-08e35e76-d08d-4cd9-ae2c-36aa61ca65a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ee08b16f3d74d69b52e7dd18902ac16'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:20:55.679147 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 856/1713] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:20:55 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:20:55.680603 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa80c2e1-37ae-42cc-9979-21714a873bd8 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:55.680897 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa80c2e1-37ae-42cc-9979-21714a873bd8 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:55.681474 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa80c2e1-37ae-42cc-9979-21714a873bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:55.714050 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-08e35e76-d08d-4cd9-ae2c-36aa61ca65a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ee08b16f3d74d69b52e7dd18902ac16'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:20:55.715843 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-08e35e76-d08d-4cd9-ae2c-36aa61ca65a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ee08b16f3d74d69b52e7dd18902ac16'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:20:55.716740 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-08e35e76-d08d-4cd9-ae2c-36aa61ca65a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ee08b16f3d74d69b52e7dd18902ac16'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:20:55.793172 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-aa80c2e1-37ae-42cc-9979-21714a873bd8 None None] MFA Rules not processed for user `7417a54046e34e899d7b03f2f2314deb`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:20:55.793367 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 858/1714] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:20:55 2022] DELETE /identity/v3/users/5ee08b16f3d74d69b52e7dd18902ac16 => generated 0 bytes in 615 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:20:55.794772 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fedb39-e1b0-46a4-a7a3-8e6f18ecf065 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:55.795541 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fedb39-e1b0-46a4-a7a3-8e6f18ecf065 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:55.796376 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fedb39-e1b0-46a4-a7a3-8e6f18ecf065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:55.799713 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-aa80c2e1-37ae-42cc-9979-21714a873bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:55.858348 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-60fedb39-e1b0-46a4-a7a3-8e6f18ecf065 None None] MFA Rules not processed for user `7f950b5cc5fa44c988b945e9cd553874`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:20:55.860613 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-60fedb39-e1b0-46a4-a7a3-8e6f18ecf065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:55.973312 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 857/1715] 10.222.0.32 () {58 vars in 1015 bytes} [Thu Jun 2 19:20:55 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:20:55.974392 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa80c2e1-37ae-42cc-9979-21714a873bd8 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:55.976987 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-aa80c2e1-37ae-42cc-9979-21714a873bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:56.114596 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 859/1716] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:20:55 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:20:56.122690 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce7cbd-888b-4082-a557-11ad02acdb34 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:56.123346 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce7cbd-888b-4082-a557-11ad02acdb34 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:56.123770 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce7cbd-888b-4082-a557-11ad02acdb34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:56.183160 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa80c2e1-37ae-42cc-9979-21714a873bd8 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:56.196528 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4884f247-fcf2-4859-993b-78e091f62f77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:56.197994 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4884f247-fcf2-4859-993b-78e091f62f77 service cinder] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:56.198430 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4884f247-fcf2-4859-993b-78e091f62f77 service cinder] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:56.198867 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4884f247-fcf2-4859-993b-78e091f62f77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:56.200961 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-d0ce7cbd-888b-4082-a557-11ad02acdb34 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:20:56.202544 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-d0ce7cbd-888b-4082-a557-11ad02acdb34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:56.206863 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-4884f247-fcf2-4859-993b-78e091f62f77 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:56.364534 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:20:56.378447 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 860/1717] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:20:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:20:56.380865 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07164c1-4c12-4b9d-895a-07be13c97cda None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:56.381314 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07164c1-4c12-4b9d-895a-07be13c97cda None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:56.381923 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07164c1-4c12-4b9d-895a-07be13c97cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:56.414194 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4884f247-fcf2-4859-993b-78e091f62f77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:56.415784 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4884f247-fcf2-4859-993b-78e091f62f77 service cinder] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:56.422801 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-c07164c1-4c12-4b9d-895a-07be13c97cda None None] MFA Rules not processed for user `7f950b5cc5fa44c988b945e9cd553874`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:20:56.424684 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-c07164c1-4c12-4b9d-895a-07be13c97cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:56.436747 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 861/1718] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:20:56 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:20:56.437922 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07164c1-4c12-4b9d-895a-07be13c97cda None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:56.439998 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-c07164c1-4c12-4b9d-895a-07be13c97cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:56.467992 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 858/1719] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:20:55 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:20:56.468109 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07164c1-4c12-4b9d-895a-07be13c97cda None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:56.468446 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4884f247-fcf2-4859-993b-78e091f62f77 service cinder] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:56.475653 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4497aab6-f563-436b-baa4-16b2b4bcb79c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:56.476349 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4497aab6-f563-436b-baa4-16b2b4bcb79c service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:56.478510 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-4884f247-fcf2-4859-993b-78e091f62f77 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:56.480275 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4497aab6-f563-436b-baa4-16b2b4bcb79c service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:56.480571 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4497aab6-f563-436b-baa4-16b2b4bcb79c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:56.522302 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4497aab6-f563-436b-baa4-16b2b4bcb79c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:56.526944 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4497aab6-f563-436b-baa4-16b2b4bcb79c service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:56.566040 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 862/1720] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:20:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:20:56.635659 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:20:56.741463 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4884f247-fcf2-4859-993b-78e091f62f77 service cinder] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:56.762887 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd13451-56da-478e-ad4e-5fa0a77e8a13 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:56.762887 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd13451-56da-478e-ad4e-5fa0a77e8a13 service nova] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:56.763340 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd13451-56da-478e-ad4e-5fa0a77e8a13 service nova] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:56.763444 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd13451-56da-478e-ad4e-5fa0a77e8a13 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:56.805059 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd13451-56da-478e-ad4e-5fa0a77e8a13 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:56.811079 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd13451-56da-478e-ad4e-5fa0a77e8a13 service nova] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:56.841008 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 859/1721] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:20:56 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:20:56.873538 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4497aab6-f563-436b-baa4-16b2b4bcb79c service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:56.889146 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4497aab6-f563-436b-baa4-16b2b4bcb79c service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:56.905021 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37c7bb0-6ab8-459a-9f17-7375e40f730b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:56.916475 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37c7bb0-6ab8-459a-9f17-7375e40f730b service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:56.947383 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37c7bb0-6ab8-459a-9f17-7375e40f730b service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:56.948728 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37c7bb0-6ab8-459a-9f17-7375e40f730b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:56.989872 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37c7bb0-6ab8-459a-9f17-7375e40f730b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:56.995652 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37c7bb0-6ab8-459a-9f17-7375e40f730b service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:57.052278 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 863/1722] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:20:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:20:58.040970 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd13451-56da-478e-ad4e-5fa0a77e8a13 service nova] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:58.042558 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-2cd13451-56da-478e-ad4e-5fa0a77e8a13 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:58.080122 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd13451-56da-478e-ad4e-5fa0a77e8a13 service nova] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:58.105023 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50db2566-de93-469c-994b-ac64e591c63d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:58.106340 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db2566-de93-469c-994b-ac64e591c63d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:58.106943 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db2566-de93-469c-994b-ac64e591c63d admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:58.107426 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db2566-de93-469c-994b-ac64e591c63d admin admin] PATH_INFO: `/v3/users/93eab725113c47e1aadbe00b41b86660` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:58.108977 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50db2566-de93-469c-994b-ac64e591c63d admin admin] RBAC: Authorizing `identity:delete_user(user_id=93eab725113c47e1aadbe00b41b86660)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:58.111840 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50db2566-de93-469c-994b-ac64e591c63d admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:58.406217 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-50db2566-de93-469c-994b-ac64e591c63d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93eab725113c47e1aadbe00b41b86660'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:20:58.437850 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-50db2566-de93-469c-994b-ac64e591c63d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93eab725113c47e1aadbe00b41b86660'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:20:58.437850 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-50db2566-de93-469c-994b-ac64e591c63d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93eab725113c47e1aadbe00b41b86660'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:20:58.514455 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-50db2566-de93-469c-994b-ac64e591c63d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93eab725113c47e1aadbe00b41b86660'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:20:58.515136 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-50db2566-de93-469c-994b-ac64e591c63d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93eab725113c47e1aadbe00b41b86660'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:20:58.556488 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 860/1723] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:20:58 2022] DELETE /identity/v3/users/93eab725113c47e1aadbe00b41b86660 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:20:58.560882 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0ac89b-6d73-41f7-80c5-f2176bda5734 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:58.561773 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0ac89b-6d73-41f7-80c5-f2176bda5734 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:58.562207 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0ac89b-6d73-41f7-80c5-f2176bda5734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:58.595107 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-eb0ac89b-6d73-41f7-80c5-f2176bda5734 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:20:58.596866 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-eb0ac89b-6d73-41f7-80c5-f2176bda5734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:58.838619 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 864/1724] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:20:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:20:58.860729 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50db2566-de93-469c-994b-ac64e591c63d admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:58.864957 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-50db2566-de93-469c-994b-ac64e591c63d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:58.947676 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:20:59.012119 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50db2566-de93-469c-994b-ac64e591c63d admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:59.028954 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a2acb7-969b-4d97-8443-9b740870403e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:59.033717 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a2acb7-969b-4d97-8443-9b740870403e service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:59.033717 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a2acb7-969b-4d97-8443-9b740870403e service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:59.033717 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a2acb7-969b-4d97-8443-9b740870403e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:59.066235 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a2acb7-969b-4d97-8443-9b740870403e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:59.068105 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a2acb7-969b-4d97-8443-9b740870403e service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:20:59.112900 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 861/1725] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:20:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:20:59.426090 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f57cd8-4d38-4aa6-85a4-687ad37654f0 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:59.426693 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f57cd8-4d38-4aa6-85a4-687ad37654f0 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:59.427416 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f57cd8-4d38-4aa6-85a4-687ad37654f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:59.441438 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:20:59.481848 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-47f57cd8-4d38-4aa6-85a4-687ad37654f0 None None] MFA Rules not processed for user `a0681f3eff324262b4289bf6d8f4c311`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:20:59.489313 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-47f57cd8-4d38-4aa6-85a4-687ad37654f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:59.567500 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a2acb7-969b-4d97-8443-9b740870403e service neutron] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:59.574057 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-47a2acb7-969b-4d97-8443-9b740870403e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:59.614542 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a2acb7-969b-4d97-8443-9b740870403e service neutron] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:20:59.630464 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2296cf9-e884-425e-adcb-2e106fd56b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:20:59.634374 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2296cf9-e884-425e-adcb-2e106fd56b5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:20:59.638054 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2296cf9-e884-425e-adcb-2e106fd56b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:20:59.653971 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 865/1726] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:20:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 19:20:59.661478 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2296cf9-e884-425e-adcb-2e106fd56b5d admin admin] PATH_INFO: `/v3/projects/8ffb28cd3ad84f31947362c2748336eb` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:20:59.669515 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2296cf9-e884-425e-adcb-2e106fd56b5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ffb28cd3ad84f31947362c2748336eb)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:20:59.670781 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f57cd8-4d38-4aa6-85a4-687ad37654f0 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:20:59.676122 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-47f57cd8-4d38-4aa6-85a4-687ad37654f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:20:59.679517 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2296cf9-e884-425e-adcb-2e106fd56b5d admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:00.003509 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-f2296cf9-e884-425e-adcb-2e106fd56b5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ffb28cd3ad84f31947362c2748336eb', 'request_id': 'req-f2296cf9-e884-425e-adcb-2e106fd56b5d'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:00.081864 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f57cd8-4d38-4aa6-85a4-687ad37654f0 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:00.112970 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-f2296cf9-e884-425e-adcb-2e106fd56b5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ffb28cd3ad84f31947362c2748336eb', 'request_id': 'req-f2296cf9-e884-425e-adcb-2e106fd56b5d'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:00.143514 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-f2296cf9-e884-425e-adcb-2e106fd56b5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ffb28cd3ad84f31947362c2748336eb', 'request_id': 'req-f2296cf9-e884-425e-adcb-2e106fd56b5d'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:00.185849 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-f2296cf9-e884-425e-adcb-2e106fd56b5d admin admin] The token cache is being invalidate because project 8ffb28cd3ad84f31947362c2748336eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:21:00.186146 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-f2296cf9-e884-425e-adcb-2e106fd56b5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:00.188983 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-f2296cf9-e884-425e-adcb-2e106fd56b5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:00.208865 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 862/1727] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:20:59 2022] DELETE /identity/v3/projects/8ffb28cd3ad84f31947362c2748336eb => generated 0 bytes in 642 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:00.209934 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2296cf9-e884-425e-adcb-2e106fd56b5d admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:00.217620 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-f2296cf9-e884-425e-adcb-2e106fd56b5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:00.387702 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba72209-c471-44bd-97da-65e1e2f8fe23 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:00.388612 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba72209-c471-44bd-97da-65e1e2f8fe23 service nova] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:00.388940 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba72209-c471-44bd-97da-65e1e2f8fe23 service nova] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:00.389183 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba72209-c471-44bd-97da-65e1e2f8fe23 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:00.398673 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-8ba72209-c471-44bd-97da-65e1e2f8fe23 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:00.511602 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:00.543722 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2296cf9-e884-425e-adcb-2e106fd56b5d admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:00.558257 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e31a0bc-d298-48ef-a634-4193b8477aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:00.559805 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e31a0bc-d298-48ef-a634-4193b8477aba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:00.560361 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e31a0bc-d298-48ef-a634-4193b8477aba admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:00.560710 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e31a0bc-d298-48ef-a634-4193b8477aba admin admin] PATH_INFO: `/v3/users/9ac7b875483e45ebbffc4961d766eea1` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:00.562258 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e31a0bc-d298-48ef-a634-4193b8477aba admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ac7b875483e45ebbffc4961d766eea1)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:00.564927 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e31a0bc-d298-48ef-a634-4193b8477aba admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:00.639230 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba72209-c471-44bd-97da-65e1e2f8fe23 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:00.640980 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba72209-c471-44bd-97da-65e1e2f8fe23 service nova] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:00.685407 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 866/1728] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:20:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 1015 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:21:00.686442 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba72209-c471-44bd-97da-65e1e2f8fe23 service nova] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:00.690376 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-8ba72209-c471-44bd-97da-65e1e2f8fe23 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:00.733238 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba72209-c471-44bd-97da-65e1e2f8fe23 service nova] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:00.759626 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0227c44c-3970-4e37-8b53-e44278ad72f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:00.759626 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0227c44c-3970-4e37-8b53-e44278ad72f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:00.759626 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0227c44c-3970-4e37-8b53-e44278ad72f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:00.760266 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0227c44c-3970-4e37-8b53-e44278ad72f4 admin admin] PATH_INFO: `/v3/users/cf952a948ff344e1ac373c66a48b0ca8` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:00.768461 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0227c44c-3970-4e37-8b53-e44278ad72f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf952a948ff344e1ac373c66a48b0ca8)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:00.775011 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0227c44c-3970-4e37-8b53-e44278ad72f4 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:00.914988 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-0e31a0bc-d298-48ef-a634-4193b8477aba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ac7b875483e45ebbffc4961d766eea1'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:00.951264 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-0e31a0bc-d298-48ef-a634-4193b8477aba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ac7b875483e45ebbffc4961d766eea1'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:00.952799 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-0e31a0bc-d298-48ef-a634-4193b8477aba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ac7b875483e45ebbffc4961d766eea1'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:00.976004 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:01.038669 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-0e31a0bc-d298-48ef-a634-4193b8477aba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ac7b875483e45ebbffc4961d766eea1'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:01.039547 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-0e31a0bc-d298-48ef-a634-4193b8477aba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ac7b875483e45ebbffc4961d766eea1'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:01.068289 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-0227c44c-3970-4e37-8b53-e44278ad72f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf952a948ff344e1ac373c66a48b0ca8'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:01.123835 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-0227c44c-3970-4e37-8b53-e44278ad72f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf952a948ff344e1ac373c66a48b0ca8'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:01.123996 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 863/1729] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:21:00 2022] DELETE /identity/v3/users/9ac7b875483e45ebbffc4961d766eea1 => generated 0 bytes in 912 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:01.128484 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7093d941-9088-40d5-a089-d46bf458c57e None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:01.129130 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7093d941-9088-40d5-a089-d46bf458c57e None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:01.129768 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7093d941-9088-40d5-a089-d46bf458c57e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:01.206046 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-7093d941-9088-40d5-a089-d46bf458c57e None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:01.212969 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-7093d941-9088-40d5-a089-d46bf458c57e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:01.244243 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-0227c44c-3970-4e37-8b53-e44278ad72f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf952a948ff344e1ac373c66a48b0ca8'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:01.245152 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-0227c44c-3970-4e37-8b53-e44278ad72f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf952a948ff344e1ac373c66a48b0ca8'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:01.245973 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-0227c44c-3970-4e37-8b53-e44278ad72f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf952a948ff344e1ac373c66a48b0ca8'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:01.274570 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 867/1730] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:21:00 2022] DELETE /identity/v3/users/cf952a948ff344e1ac373c66a48b0ca8 => generated 0 bytes in 585 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:01.295412 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55fd6f6-5bfe-4d8b-bd6a-eafbdcfde2b4 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:01.295705 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55fd6f6-5bfe-4d8b-bd6a-eafbdcfde2b4 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:01.296005 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55fd6f6-5bfe-4d8b-bd6a-eafbdcfde2b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:01.421746 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-c55fd6f6-5bfe-4d8b-bd6a-eafbdcfde2b4 None None] MFA Rules not processed for user `df62e9ab4d044dcaab00c5d76416e3e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:01.421746 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-c55fd6f6-5bfe-4d8b-bd6a-eafbdcfde2b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:01.473390 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 864/1731] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:21:01.475123 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c923ec01-5745-4fa9-a599-dcb9e8e98e9c None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:01.475445 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c923ec01-5745-4fa9-a599-dcb9e8e98e9c None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:01.475743 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c923ec01-5745-4fa9-a599-dcb9e8e98e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:01.529721 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-c923ec01-5745-4fa9-a599-dcb9e8e98e9c None None] MFA Rules not processed for user `7f950b5cc5fa44c988b945e9cd553874`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:01.532819 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-c923ec01-5745-4fa9-a599-dcb9e8e98e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:01.627156 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 868/1732] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:01 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:21:01.628291 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55fd6f6-5bfe-4d8b-bd6a-eafbdcfde2b4 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:01.631809 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-c55fd6f6-5bfe-4d8b-bd6a-eafbdcfde2b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:01.735527 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 865/1733] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:01 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:21:01.735527 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c923ec01-5745-4fa9-a599-dcb9e8e98e9c None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:01.737251 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-c923ec01-5745-4fa9-a599-dcb9e8e98e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:01.746998 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:01.828609 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55fd6f6-5bfe-4d8b-bd6a-eafbdcfde2b4 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:01.842933 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f96052a-b540-443b-94c8-458f3af02893 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:01.844823 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f96052a-b540-443b-94c8-458f3af02893 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:01.844823 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f96052a-b540-443b-94c8-458f3af02893 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:01.844823 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f96052a-b540-443b-94c8-458f3af02893 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:02.066915 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f96052a-b540-443b-94c8-458f3af02893 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:02.068852 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f96052a-b540-443b-94c8-458f3af02893 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:02.074583 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c923ec01-5745-4fa9-a599-dcb9e8e98e9c None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:02.088853 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a094a2-c0bf-408e-a945-c184d85bbffd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:02.089555 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a094a2-c0bf-408e-a945-c184d85bbffd service nova] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:02.089846 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a094a2-c0bf-408e-a945-c184d85bbffd service nova] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:02.092390 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a094a2-c0bf-408e-a945-c184d85bbffd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:02.094852 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 869/1734] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:21:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:21:02.096428 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f96052a-b540-443b-94c8-458f3af02893 service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:02.101265 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-0f96052a-b540-443b-94c8-458f3af02893 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:02.145790 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a094a2-c0bf-408e-a945-c184d85bbffd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:02.151930 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f96052a-b540-443b-94c8-458f3af02893 service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:02.153010 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a094a2-c0bf-408e-a945-c184d85bbffd service nova] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:02.166447 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d2d19d-8eef-40a0-8e31-5e3889e23a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:02.170509 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d2d19d-8eef-40a0-8e31-5e3889e23a7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:02.170509 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d2d19d-8eef-40a0-8e31-5e3889e23a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:02.170509 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d2d19d-8eef-40a0-8e31-5e3889e23a7e admin admin] PATH_INFO: `/v3/projects/7a09abe3e85c4c83a6401029a539da4b` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:02.173971 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d2d19d-8eef-40a0-8e31-5e3889e23a7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a09abe3e85c4c83a6401029a539da4b)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:02.181163 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d2d19d-8eef-40a0-8e31-5e3889e23a7e admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:02.195436 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 866/1735] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:21:01 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:21:02.488450 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0721d0-f989-4b17-a7e6-03c6c290bf72 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:02.488733 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0721d0-f989-4b17-a7e6-03c6c290bf72 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:02.489177 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0721d0-f989-4b17-a7e6-03c6c290bf72 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:02.491038 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 867/1736] 10.222.0.33 () {54 vars in 921 bytes} [Thu Jun 2 19:21:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:21:02.512437 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-b4d2d19d-8eef-40a0-8e31-5e3889e23a7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a09abe3e85c4c83a6401029a539da4b', 'request_id': 'req-b4d2d19d-8eef-40a0-8e31-5e3889e23a7e'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:02.543077 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ccc3c-76cb-4dd3-be5b-a3da0113e41f None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:02.543378 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ccc3c-76cb-4dd3-be5b-a3da0113e41f None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:02.543941 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ccc3c-76cb-4dd3-be5b-a3da0113e41f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:02.587102 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-619ccc3c-76cb-4dd3-be5b-a3da0113e41f None None] MFA Rules not processed for user `70664d5cbdee47ee89c120761939cc75`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:02.588744 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-619ccc3c-76cb-4dd3-be5b-a3da0113e41f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:02.600896 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:02.628892 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-b4d2d19d-8eef-40a0-8e31-5e3889e23a7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a09abe3e85c4c83a6401029a539da4b', 'request_id': 'req-b4d2d19d-8eef-40a0-8e31-5e3889e23a7e'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:02.638143 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-b4d2d19d-8eef-40a0-8e31-5e3889e23a7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a09abe3e85c4c83a6401029a539da4b', 'request_id': 'req-b4d2d19d-8eef-40a0-8e31-5e3889e23a7e'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:02.717309 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-b4d2d19d-8eef-40a0-8e31-5e3889e23a7e admin admin] The token cache is being invalidate because project 7a09abe3e85c4c83a6401029a539da4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:21:02.718186 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-b4d2d19d-8eef-40a0-8e31-5e3889e23a7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:02.722614 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-b4d2d19d-8eef-40a0-8e31-5e3889e23a7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:02.732384 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 870/1737] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:21:02 2022] DELETE /identity/v3/projects/7a09abe3e85c4c83a6401029a539da4b => generated 0 bytes in 636 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:02.738638 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d2d19d-8eef-40a0-8e31-5e3889e23a7e admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:02.742668 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-b4d2d19d-8eef-40a0-8e31-5e3889e23a7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:02.933222 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 868/1738] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Jun 2 19:21:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 391 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:21:03.199125 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d2d19d-8eef-40a0-8e31-5e3889e23a7e admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:03.238109 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063e8dc0-66aa-4f42-89c8-22427873d191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:03.243277 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e8dc0-66aa-4f42-89c8-22427873d191 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:03.244328 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e8dc0-66aa-4f42-89c8-22427873d191 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:03.244328 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e8dc0-66aa-4f42-89c8-22427873d191 admin admin] PATH_INFO: `/v3/users/a3ac833d0e8c4202a0d60ec731d9899b` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:03.258348 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063e8dc0-66aa-4f42-89c8-22427873d191 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3ac833d0e8c4202a0d60ec731d9899b)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:03.267636 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063e8dc0-66aa-4f42-89c8-22427873d191 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:03.277718 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:03.397851 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ccc3c-76cb-4dd3-be5b-a3da0113e41f None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:03.399933 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-619ccc3c-76cb-4dd3-be5b-a3da0113e41f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:03.461827 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ccc3c-76cb-4dd3-be5b-a3da0113e41f None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:03.478859 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343a139d-ad70-4718-b8db-7a0cb9bbf0fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:03.480119 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a139d-ad70-4718-b8db-7a0cb9bbf0fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:03.480646 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a139d-ad70-4718-b8db-7a0cb9bbf0fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:03.485980 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a139d-ad70-4718-b8db-7a0cb9bbf0fb admin admin] PATH_INFO: `/v3/projects/11cb34b81f8b4892a79e034de933c4ea` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:03.493172 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343a139d-ad70-4718-b8db-7a0cb9bbf0fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=11cb34b81f8b4892a79e034de933c4ea)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:03.499512 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343a139d-ad70-4718-b8db-7a0cb9bbf0fb admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:03.675925 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-063e8dc0-66aa-4f42-89c8-22427873d191 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3ac833d0e8c4202a0d60ec731d9899b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:03.704324 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-063e8dc0-66aa-4f42-89c8-22427873d191 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3ac833d0e8c4202a0d60ec731d9899b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:03.739861 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-343a139d-ad70-4718-b8db-7a0cb9bbf0fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11cb34b81f8b4892a79e034de933c4ea', 'request_id': 'req-343a139d-ad70-4718-b8db-7a0cb9bbf0fb'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:03.821437 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-063e8dc0-66aa-4f42-89c8-22427873d191 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3ac833d0e8c4202a0d60ec731d9899b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:03.822186 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-063e8dc0-66aa-4f42-89c8-22427873d191 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3ac833d0e8c4202a0d60ec731d9899b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:03.822872 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-063e8dc0-66aa-4f42-89c8-22427873d191 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3ac833d0e8c4202a0d60ec731d9899b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:03.895556 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-343a139d-ad70-4718-b8db-7a0cb9bbf0fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11cb34b81f8b4892a79e034de933c4ea', 'request_id': 'req-343a139d-ad70-4718-b8db-7a0cb9bbf0fb'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:03.896403 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 871/1739] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:21:02 2022] DELETE /identity/v3/users/a3ac833d0e8c4202a0d60ec731d9899b => generated 0 bytes in 1155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:03.906442 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-343a139d-ad70-4718-b8db-7a0cb9bbf0fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11cb34b81f8b4892a79e034de933c4ea', 'request_id': 'req-343a139d-ad70-4718-b8db-7a0cb9bbf0fb'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:03.942506 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-343a139d-ad70-4718-b8db-7a0cb9bbf0fb admin admin] The token cache is being invalidate because project 11cb34b81f8b4892a79e034de933c4ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:21:03.942827 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-343a139d-ad70-4718-b8db-7a0cb9bbf0fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:03.945713 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-343a139d-ad70-4718-b8db-7a0cb9bbf0fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:03.950979 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 869/1740] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:21:03 2022] DELETE /identity/v3/projects/11cb34b81f8b4892a79e034de933c4ea => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 02 19:21:03.952901 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d8ca3-da87-44e5-88b6-89b03913b1b5 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:03.953310 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d8ca3-da87-44e5-88b6-89b03913b1b5 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:03.953698 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d8ca3-da87-44e5-88b6-89b03913b1b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:04.011761 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-ae5d8ca3-da87-44e5-88b6-89b03913b1b5 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:04.013506 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-ae5d8ca3-da87-44e5-88b6-89b03913b1b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:04.257098 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 872/1741] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:21:04.260588 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfb4567-6306-404a-a52c-57ba91d21ad1 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:04.263267 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfb4567-6306-404a-a52c-57ba91d21ad1 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:04.263571 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfb4567-6306-404a-a52c-57ba91d21ad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:04.322897 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-fbfb4567-6306-404a-a52c-57ba91d21ad1 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:04.324170 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-fbfb4567-6306-404a-a52c-57ba91d21ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:04.344845 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 870/1742] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:21:04.346735 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5d8ca3-da87-44e5-88b6-89b03913b1b5 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:04.366441 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5d8ca3-da87-44e5-88b6-89b03913b1b5 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:04.395967 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8095024f-4bfb-4a54-8245-5e83d4cffd52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:04.397020 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8095024f-4bfb-4a54-8245-5e83d4cffd52 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:04.397274 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8095024f-4bfb-4a54-8245-5e83d4cffd52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:04.397559 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8095024f-4bfb-4a54-8245-5e83d4cffd52 admin admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:04.406552 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8095024f-4bfb-4a54-8245-5e83d4cffd52 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:21:04.407296 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8095024f-4bfb-4a54-8245-5e83d4cffd52 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:04.411308 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8095024f-4bfb-4a54-8245-5e83d4cffd52 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:04.457954 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 873/1743] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 2 19:21:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:21:04.460792 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfb4567-6306-404a-a52c-57ba91d21ad1 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:04.476737 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfb4567-6306-404a-a52c-57ba91d21ad1 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:04.515730 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ab1a97-4686-4b63-a097-39b9ff97e861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:04.516973 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab1a97-4686-4b63-a097-39b9ff97e861 admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:04.517272 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab1a97-4686-4b63-a097-39b9ff97e861 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:04.518093 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab1a97-4686-4b63-a097-39b9ff97e861 admin admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:04.523874 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ab1a97-4686-4b63-a097-39b9ff97e861 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:04.526251 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ab1a97-4686-4b63-a097-39b9ff97e861 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:04.528867 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8095024f-4bfb-4a54-8245-5e83d4cffd52 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:04.530807 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-8095024f-4bfb-4a54-8245-5e83d4cffd52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:04.763891 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 871/1744] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:21:04 2022] POST /identity/v3/projects => generated 371 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:21:04.766899 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8095024f-4bfb-4a54-8245-5e83d4cffd52 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:04.784096 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ab1a97-4686-4b63-a097-39b9ff97e861 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:04.795522 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:05.052905 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:05.072132 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ab1a97-4686-4b63-a097-39b9ff97e861 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:05.072742 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9811748-4f6a-4543-82c8-45ea8aa239ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:05.076563 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9811748-4f6a-4543-82c8-45ea8aa239ab service nova] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:05.076924 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9811748-4f6a-4543-82c8-45ea8aa239ab service nova] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:05.077222 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9811748-4f6a-4543-82c8-45ea8aa239ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:05.083450 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-f9811748-4f6a-4543-82c8-45ea8aa239ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:05.096947 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03211cbc-36df-42a3-b570-059062ec820f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:05.097743 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03211cbc-36df-42a3-b570-059062ec820f admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:05.098032 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03211cbc-36df-42a3-b570-059062ec820f admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:05.100460 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03211cbc-36df-42a3-b570-059062ec820f admin admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:05.101226 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03211cbc-36df-42a3-b570-059062ec820f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:05.111177 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03211cbc-36df-42a3-b570-059062ec820f admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:05.131237 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9811748-4f6a-4543-82c8-45ea8aa239ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:05.136158 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9811748-4f6a-4543-82c8-45ea8aa239ab service nova] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:05.171834 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 874/1745] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:21:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 644 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:21:05.219973 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9811748-4f6a-4543-82c8-45ea8aa239ab service nova] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:05.234142 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 872/1746] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:21:04 2022] POST /identity/v3/users => generated 286 bytes in 467 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:21:05.250638 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9811748-4f6a-4543-82c8-45ea8aa239ab service nova] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:05.268389 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86a4f6d-9d8e-4aea-b11a-ad8fad3e23a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:05.269055 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86a4f6d-9d8e-4aea-b11a-ad8fad3e23a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:05.271439 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86a4f6d-9d8e-4aea-b11a-ad8fad3e23a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:05.271753 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86a4f6d-9d8e-4aea-b11a-ad8fad3e23a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:05.273650 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86a4f6d-9d8e-4aea-b11a-ad8fad3e23a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:05.275345 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86a4f6d-9d8e-4aea-b11a-ad8fad3e23a0 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:05.331002 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03211cbc-36df-42a3-b570-059062ec820f admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:05.331111 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 875/1747] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:21:05 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:21:05.331197 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03211cbc-36df-42a3-b570-059062ec820f admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:05.360017 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a2e324-3c2a-45a8-ac24-fa7d68427666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:05.360917 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a2e324-3c2a-45a8-ac24-fa7d68427666 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:05.363194 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a2e324-3c2a-45a8-ac24-fa7d68427666 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:05.364873 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a2e324-3c2a-45a8-ac24-fa7d68427666 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:05.368057 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a2e324-3c2a-45a8-ac24-fa7d68427666 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:05.373091 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a2e324-3c2a-45a8-ac24-fa7d68427666 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:05.413770 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86a4f6d-9d8e-4aea-b11a-ad8fad3e23a0 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:05.426311 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 873/1748] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:21:05 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:21:05.432687 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86a4f6d-9d8e-4aea-b11a-ad8fad3e23a0 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:05.444581 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b12b780-208e-4a79-a51f-c6161b831b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:05.445627 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b12b780-208e-4a79-a51f-c6161b831b49 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:05.445920 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b12b780-208e-4a79-a51f-c6161b831b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:05.446205 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b12b780-208e-4a79-a51f-c6161b831b49 admin admin] PATH_INFO: `/v3/projects/2cb2e1eb3a5e4c0bab176a8e0be12dc9/users/72558099db31474292377bab47035911/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:05.454649 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a2e324-3c2a-45a8-ac24-fa7d68427666 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:05.457588 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-f0a2e324-3c2a-45a8-ac24-fa7d68427666 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:05.489378 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a2e324-3c2a-45a8-ac24-fa7d68427666 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:05.519489 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b12b780-208e-4a79-a51f-c6161b831b49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cb2e1eb3a5e4c0bab176a8e0be12dc9, user_id=72558099db31474292377bab47035911, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:05.520344 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dce881f-a222-4e8d-a14a-9690c15496f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:05.522027 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce881f-a222-4e8d-a14a-9690c15496f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:05.522315 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce881f-a222-4e8d-a14a-9690c15496f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:05.531735 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b12b780-208e-4a79-a51f-c6161b831b49 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:05.534002 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce881f-a222-4e8d-a14a-9690c15496f7 admin admin] PATH_INFO: `/v3/users/c70eaf14c2c04d7596359dd7a1ecc16e` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:05.538638 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dce881f-a222-4e8d-a14a-9690c15496f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c70eaf14c2c04d7596359dd7a1ecc16e)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:05.541910 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dce881f-a222-4e8d-a14a-9690c15496f7 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:05.641899 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 876/1749] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:21:05 2022] PUT /identity/v3/projects/2cb2e1eb3a5e4c0bab176a8e0be12dc9/users/72558099db31474292377bab47035911/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 02 19:21:05.648254 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b75b8a6-7c2d-4883-aafa-cd322c2687a1 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:05.648554 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b75b8a6-7c2d-4883-aafa-cd322c2687a1 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:05.650178 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b75b8a6-7c2d-4883-aafa-cd322c2687a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:05.687794 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-9b75b8a6-7c2d-4883-aafa-cd322c2687a1 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:05.694388 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-9b75b8a6-7c2d-4883-aafa-cd322c2687a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:05.837389 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-9dce881f-a222-4e8d-a14a-9690c15496f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c70eaf14c2c04d7596359dd7a1ecc16e'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:05.852603 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-9dce881f-a222-4e8d-a14a-9690c15496f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c70eaf14c2c04d7596359dd7a1ecc16e'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:05.856462 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-9dce881f-a222-4e8d-a14a-9690c15496f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c70eaf14c2c04d7596359dd7a1ecc16e'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:05.871781 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 877/1750] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:21:05.871781 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b75b8a6-7c2d-4883-aafa-cd322c2687a1 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:05.871781 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-9b75b8a6-7c2d-4883-aafa-cd322c2687a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:05.926582 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b75b8a6-7c2d-4883-aafa-cd322c2687a1 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:05.953950 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-9dce881f-a222-4e8d-a14a-9690c15496f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c70eaf14c2c04d7596359dd7a1ecc16e'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:05.955126 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-9dce881f-a222-4e8d-a14a-9690c15496f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c70eaf14c2c04d7596359dd7a1ecc16e'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:05.969924 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f069e890-66a6-409d-b02c-656d358c2d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:05.975441 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f069e890-66a6-409d-b02c-656d358c2d42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:05.976986 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f069e890-66a6-409d-b02c-656d358c2d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:05.977343 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f069e890-66a6-409d-b02c-656d358c2d42 admin admin] PATH_INFO: `/v3/projects/68e10a6c688f4084be40b2eaf52a0784` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:05.983126 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f069e890-66a6-409d-b02c-656d358c2d42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68e10a6c688f4084be40b2eaf52a0784)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:05.991535 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f069e890-66a6-409d-b02c-656d358c2d42 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:05.992902 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 874/1751] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:21:05 2022] DELETE /identity/v3/users/c70eaf14c2c04d7596359dd7a1ecc16e => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:05.993940 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dce881f-a222-4e8d-a14a-9690c15496f7 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:05.996399 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-9dce881f-a222-4e8d-a14a-9690c15496f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:06.157994 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dce881f-a222-4e8d-a14a-9690c15496f7 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:06.169345 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410dcc76-b42c-48ca-87b6-36e28e2cdffb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:06.170910 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410dcc76-b42c-48ca-87b6-36e28e2cdffb service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:06.171210 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410dcc76-b42c-48ca-87b6-36e28e2cdffb service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:06.171618 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410dcc76-b42c-48ca-87b6-36e28e2cdffb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:06.269881 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f069e890-66a6-409d-b02c-656d358c2d42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68e10a6c688f4084be40b2eaf52a0784', 'request_id': 'req-f069e890-66a6-409d-b02c-656d358c2d42'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:06.292264 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:06.354690 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f069e890-66a6-409d-b02c-656d358c2d42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68e10a6c688f4084be40b2eaf52a0784', 'request_id': 'req-f069e890-66a6-409d-b02c-656d358c2d42'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:06.361067 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f069e890-66a6-409d-b02c-656d358c2d42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68e10a6c688f4084be40b2eaf52a0784', 'request_id': 'req-f069e890-66a6-409d-b02c-656d358c2d42'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:06.362384 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:06.376022 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410dcc76-b42c-48ca-87b6-36e28e2cdffb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:06.380640 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410dcc76-b42c-48ca-87b6-36e28e2cdffb service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:06.417863 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 875/1752] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:21:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:21:06.421038 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f1dd0a-b958-4b76-a21e-50475da98906 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:06.425394 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f069e890-66a6-409d-b02c-656d358c2d42 admin admin] The token cache is being invalidate because project 68e10a6c688f4084be40b2eaf52a0784 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:21:06.426109 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f1dd0a-b958-4b76-a21e-50475da98906 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:06.427653 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f1dd0a-b958-4b76-a21e-50475da98906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:06.427777 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f069e890-66a6-409d-b02c-656d358c2d42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:06.428663 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f069e890-66a6-409d-b02c-656d358c2d42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:06.430473 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 878/1753] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:21:05 2022] DELETE /identity/v3/projects/68e10a6c688f4084be40b2eaf52a0784 => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:06.436105 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed93f40d-1366-4615-8c83-6b67f098b7f7 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:06.436489 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed93f40d-1366-4615-8c83-6b67f098b7f7 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:06.437027 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed93f40d-1366-4615-8c83-6b67f098b7f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:06.494384 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-87f1dd0a-b958-4b76-a21e-50475da98906 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:06.496491 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-87f1dd0a-b958-4b76-a21e-50475da98906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:06.510362 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-ed93f40d-1366-4615-8c83-6b67f098b7f7 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:06.511669 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-ed93f40d-1366-4615-8c83-6b67f098b7f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:06.682631 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 876/1754] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:21:06.725957 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 879/1755] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:21:06.733341 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ab3c3f-0da9-4be7-a984-e1ca0c64c9a3 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:06.733634 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ab3c3f-0da9-4be7-a984-e1ca0c64c9a3 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:06.733918 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ab3c3f-0da9-4be7-a984-e1ca0c64c9a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:06.766089 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed93f40d-1366-4615-8c83-6b67f098b7f7 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:06.766089 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-ed93f40d-1366-4615-8c83-6b67f098b7f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:06.780631 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-c1ab3c3f-0da9-4be7-a984-e1ca0c64c9a3 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:06.782550 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-c1ab3c3f-0da9-4be7-a984-e1ca0c64c9a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:06.809919 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 877/1756] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:21:06.815753 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ab3c3f-0da9-4be7-a984-e1ca0c64c9a3 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:06.835777 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ab3c3f-0da9-4be7-a984-e1ca0c64c9a3 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:06.842687 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7f5730-bb46-473f-8dcd-c3831f61c929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:06.843903 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7f5730-bb46-473f-8dcd-c3831f61c929 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:06.844387 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7f5730-bb46-473f-8dcd-c3831f61c929 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:06.845042 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7f5730-bb46-473f-8dcd-c3831f61c929 admin admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:06.846563 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7f5730-bb46-473f-8dcd-c3831f61c929 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:21:06.847170 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7f5730-bb46-473f-8dcd-c3831f61c929 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:06.850904 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7f5730-bb46-473f-8dcd-c3831f61c929 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:06.876417 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed93f40d-1366-4615-8c83-6b67f098b7f7 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:06.881224 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c65bb4-1e55-4535-aafe-50f2810f3d9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:06.881926 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c65bb4-1e55-4535-aafe-50f2810f3d9a service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:06.883255 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c65bb4-1e55-4535-aafe-50f2810f3d9a service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:06.883548 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c65bb4-1e55-4535-aafe-50f2810f3d9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:06.887777 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 878/1757] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 2 19:21:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:21:06.888692 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7f5730-bb46-473f-8dcd-c3831f61c929 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:06.893011 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-8c7f5730-bb46-473f-8dcd-c3831f61c929 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:06.900954 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c65bb4-1e55-4535-aafe-50f2810f3d9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:06.903386 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c65bb4-1e55-4535-aafe-50f2810f3d9a service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:06.917092 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7f5730-bb46-473f-8dcd-c3831f61c929 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:06.928354 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 880/1758] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:21:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 02 19:21:06.929526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757cbebf-750b-4f06-8623-5c88bd2bbf10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:06.929732 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c65bb4-1e55-4535-aafe-50f2810f3d9a service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:06.930406 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757cbebf-750b-4f06-8623-5c88bd2bbf10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:06.933083 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757cbebf-750b-4f06-8623-5c88bd2bbf10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:06.933083 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757cbebf-750b-4f06-8623-5c88bd2bbf10 admin admin] PATH_INFO: `/v3/projects/c282e21215d74158aa1891734ac7b199` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:06.943492 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757cbebf-750b-4f06-8623-5c88bd2bbf10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c282e21215d74158aa1891734ac7b199)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:06.948405 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757cbebf-750b-4f06-8623-5c88bd2bbf10 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:06.951544 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c65bb4-1e55-4535-aafe-50f2810f3d9a service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:06.956876 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7a5fe1-df95-40a7-9d66-85aff251ee7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:06.957699 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7a5fe1-df95-40a7-9d66-85aff251ee7f admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:06.957989 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7a5fe1-df95-40a7-9d66-85aff251ee7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:06.958268 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7a5fe1-df95-40a7-9d66-85aff251ee7f admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:06.959166 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7a5fe1-df95-40a7-9d66-85aff251ee7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:06.975139 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7a5fe1-df95-40a7-9d66-85aff251ee7f admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:07.056876 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 881/1759] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:21:06 2022] POST /identity/v3/projects => generated 403 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:21:07.061406 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7a5fe1-df95-40a7-9d66-85aff251ee7f admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:07.161525 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-757cbebf-750b-4f06-8623-5c88bd2bbf10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c282e21215d74158aa1891734ac7b199', 'request_id': 'req-757cbebf-750b-4f06-8623-5c88bd2bbf10'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:07.256633 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7a5fe1-df95-40a7-9d66-85aff251ee7f admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:07.262907 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-757cbebf-750b-4f06-8623-5c88bd2bbf10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c282e21215d74158aa1891734ac7b199', 'request_id': 'req-757cbebf-750b-4f06-8623-5c88bd2bbf10'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:07.276712 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-757cbebf-750b-4f06-8623-5c88bd2bbf10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c282e21215d74158aa1891734ac7b199', 'request_id': 'req-757cbebf-750b-4f06-8623-5c88bd2bbf10'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:07.280879 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:07.328586 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-757cbebf-750b-4f06-8623-5c88bd2bbf10 admin admin] The token cache is being invalidate because project c282e21215d74158aa1891734ac7b199 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:21:07.339071 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-757cbebf-750b-4f06-8623-5c88bd2bbf10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:07.339071 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-757cbebf-750b-4f06-8623-5c88bd2bbf10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:07.339071 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 879/1760] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:21:06 2022] DELETE /identity/v3/projects/c282e21215d74158aa1891734ac7b199 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:07.341213 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d86cb98-d92e-4d90-aaac-998c1b68df70 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:07.343036 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d86cb98-d92e-4d90-aaac-998c1b68df70 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:07.343762 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d86cb98-d92e-4d90-aaac-998c1b68df70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:07.377831 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-6d86cb98-d92e-4d90-aaac-998c1b68df70 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:07.380231 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-6d86cb98-d92e-4d90-aaac-998c1b68df70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:07.437117 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ec2cd4-2195-4a91-81ea-1b08bf0df003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:07.437705 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ec2cd4-2195-4a91-81ea-1b08bf0df003 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:07.438153 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ec2cd4-2195-4a91-81ea-1b08bf0df003 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:07.438671 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ec2cd4-2195-4a91-81ea-1b08bf0df003 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:07.440053 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ec2cd4-2195-4a91-81ea-1b08bf0df003 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:07.443279 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ec2cd4-2195-4a91-81ea-1b08bf0df003 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:07.561648 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:07.577924 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 882/1761] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:21:07 2022] POST /identity/v3/users => generated 302 bytes in 517 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:21:07.579211 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ec2cd4-2195-4a91-81ea-1b08bf0df003 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:07.582409 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-b2ec2cd4-2195-4a91-81ea-1b08bf0df003 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:07.596863 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 880/1762] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:21:07.614465 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccaedea-1e81-4642-ac55-8e73252bd676 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:07.621998 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccaedea-1e81-4642-ac55-8e73252bd676 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:07.621998 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccaedea-1e81-4642-ac55-8e73252bd676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:07.623489 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ec2cd4-2195-4a91-81ea-1b08bf0df003 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:07.634369 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40e30ce-c67d-467d-b713-1c25731f0371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:07.635316 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e30ce-c67d-467d-b713-1c25731f0371 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:07.636011 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e30ce-c67d-467d-b713-1c25731f0371 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:07.637165 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e30ce-c67d-467d-b713-1c25731f0371 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:07.640733 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40e30ce-c67d-467d-b713-1c25731f0371 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:07.645620 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40e30ce-c67d-467d-b713-1c25731f0371 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:07.687413 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 883/1763] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:21:07 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:21:07.690115 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40e30ce-c67d-467d-b713-1c25731f0371 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:07.700775 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-4ccaedea-1e81-4642-ac55-8e73252bd676 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:07.713601 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40e30ce-c67d-467d-b713-1c25731f0371 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:07.726740 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d835fc-f7d5-4209-ae6b-0fc9db5270ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:07.727558 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d835fc-f7d5-4209-ae6b-0fc9db5270ca admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:07.731435 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d835fc-f7d5-4209-ae6b-0fc9db5270ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:07.731435 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d835fc-f7d5-4209-ae6b-0fc9db5270ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:07.731629 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-4ccaedea-1e81-4642-ac55-8e73252bd676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:07.731717 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d835fc-f7d5-4209-ae6b-0fc9db5270ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:07.735104 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d835fc-f7d5-4209-ae6b-0fc9db5270ca admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:07.761680 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 881/1764] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:21:07.764883 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccaedea-1e81-4642-ac55-8e73252bd676 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:07.788864 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 884/1765] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:21:07 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:21:07.789048 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccaedea-1e81-4642-ac55-8e73252bd676 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:07.791788 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d835fc-f7d5-4209-ae6b-0fc9db5270ca admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:07.798380 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a46b987-f3c5-4743-b47d-c1d2adca1a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:07.799797 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a46b987-f3c5-4743-b47d-c1d2adca1a80 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:07.800336 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a46b987-f3c5-4743-b47d-c1d2adca1a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:07.801162 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a46b987-f3c5-4743-b47d-c1d2adca1a80 admin admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:07.816705 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d835fc-f7d5-4209-ae6b-0fc9db5270ca admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:07.837739 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a46b987-f3c5-4743-b47d-c1d2adca1a80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:21:07.842007 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a46b987-f3c5-4743-b47d-c1d2adca1a80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:07.852164 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0438fbb-3a1b-4bab-8a96-a36e1164d32d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:07.853164 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0438fbb-3a1b-4bab-8a96-a36e1164d32d admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:07.853445 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0438fbb-3a1b-4bab-8a96-a36e1164d32d admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:07.853746 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0438fbb-3a1b-4bab-8a96-a36e1164d32d admin admin] PATH_INFO: `/v3/projects/734454f7f1b949f3bf086a24989f8280/users/35d9904f560f455dbcc07ef0436d1a69/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:07.856785 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a46b987-f3c5-4743-b47d-c1d2adca1a80 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:07.902347 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 882/1766] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 2 19:21:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:21:07.903379 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a46b987-f3c5-4743-b47d-c1d2adca1a80 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:07.920341 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0438fbb-3a1b-4bab-8a96-a36e1164d32d admin admin] RBAC: Authorizing `identity:create_grant(project_id=734454f7f1b949f3bf086a24989f8280, user_id=35d9904f560f455dbcc07ef0436d1a69, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:07.929883 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a46b987-f3c5-4743-b47d-c1d2adca1a80 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:07.936363 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0438fbb-3a1b-4bab-8a96-a36e1164d32d admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:07.944386 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9087ec-dc26-4050-bcfe-d28e96a0151a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:07.948177 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9087ec-dc26-4050-bcfe-d28e96a0151a admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:07.955498 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9087ec-dc26-4050-bcfe-d28e96a0151a admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:07.955833 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9087ec-dc26-4050-bcfe-d28e96a0151a admin admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:07.958553 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9087ec-dc26-4050-bcfe-d28e96a0151a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:07.961165 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9087ec-dc26-4050-bcfe-d28e96a0151a admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:08.029655 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 885/1767] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:21:07 2022] PUT /identity/v3/projects/734454f7f1b949f3bf086a24989f8280/users/35d9904f560f455dbcc07ef0436d1a69/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 02 19:21:08.035608 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dffdbc-ba15-4478-9345-9bd3a0eb82e9 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:08.036097 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dffdbc-ba15-4478-9345-9bd3a0eb82e9 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:08.036619 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dffdbc-ba15-4478-9345-9bd3a0eb82e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:08.086623 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-c7dffdbc-ba15-4478-9345-9bd3a0eb82e9 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:08.087759 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-c7dffdbc-ba15-4478-9345-9bd3a0eb82e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:08.137545 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 883/1768] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:21:07 2022] POST /identity/v3/projects => generated 367 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:21:08.138936 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9087ec-dc26-4050-bcfe-d28e96a0151a admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:08.281213 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 886/1769] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:21:08.361798 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dffdbc-ba15-4478-9345-9bd3a0eb82e9 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:08.363705 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-c7dffdbc-ba15-4478-9345-9bd3a0eb82e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:08.391848 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9087ec-dc26-4050-bcfe-d28e96a0151a admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:08.415626 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4857150f-462a-410b-9243-94165ca96120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:08.415626 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4857150f-462a-410b-9243-94165ca96120 admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:08.417829 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4857150f-462a-410b-9243-94165ca96120 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:08.418174 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4857150f-462a-410b-9243-94165ca96120 admin admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:08.419111 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4857150f-462a-410b-9243-94165ca96120 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:08.421920 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4857150f-462a-410b-9243-94165ca96120 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:08.536766 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 884/1770] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:21:08 2022] POST /identity/v3/users => generated 284 bytes in 398 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:21:08.537699 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4857150f-462a-410b-9243-94165ca96120 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:08.550623 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dffdbc-ba15-4478-9345-9bd3a0eb82e9 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:08.568655 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4857150f-462a-410b-9243-94165ca96120 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:08.580871 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88a776f-3b46-44e9-b715-af6e24b09d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:08.584308 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:08.586725 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88a776f-3b46-44e9-b715-af6e24b09d54 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:08.587887 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88a776f-3b46-44e9-b715-af6e24b09d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:08.588182 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88a776f-3b46-44e9-b715-af6e24b09d54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:08.590949 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88a776f-3b46-44e9-b715-af6e24b09d54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:08.600122 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88a776f-3b46-44e9-b715-af6e24b09d54 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:08.625258 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 885/1771] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:21:08 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:21:08.630299 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88a776f-3b46-44e9-b715-af6e24b09d54 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:08.652807 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88a776f-3b46-44e9-b715-af6e24b09d54 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:08.669771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19f017a-741a-4b88-b6b2-a3314dbf389e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:08.670632 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f017a-741a-4b88-b6b2-a3314dbf389e admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:08.670948 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f017a-741a-4b88-b6b2-a3314dbf389e admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:08.671231 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f017a-741a-4b88-b6b2-a3314dbf389e admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:08.671980 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19f017a-741a-4b88-b6b2-a3314dbf389e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:08.681829 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19f017a-741a-4b88-b6b2-a3314dbf389e admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:08.737051 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 886/1772] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:21:08 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:21:08.740405 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19f017a-741a-4b88-b6b2-a3314dbf389e admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:08.778829 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19f017a-741a-4b88-b6b2-a3314dbf389e admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:08.783499 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b639a68c-66d9-4b2c-a0fd-9dfee9d43743 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:08.784345 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b639a68c-66d9-4b2c-a0fd-9dfee9d43743 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:08.784732 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b639a68c-66d9-4b2c-a0fd-9dfee9d43743 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:08.785146 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b639a68c-66d9-4b2c-a0fd-9dfee9d43743 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:08.795521 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07aa185-85c0-4a75-8fa1-c66ae73156dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:08.799179 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07aa185-85c0-4a75-8fa1-c66ae73156dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:08.799587 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07aa185-85c0-4a75-8fa1-c66ae73156dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:08.800062 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07aa185-85c0-4a75-8fa1-c66ae73156dd admin admin] PATH_INFO: `/v3/projects/d85f2cc06c80451bb5ced17157ac0271/users/8afd12d701ac4fabb3d3f18a6cc76833/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:08.815520 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b639a68c-66d9-4b2c-a0fd-9dfee9d43743 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:08.817982 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b639a68c-66d9-4b2c-a0fd-9dfee9d43743 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:08.822657 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07aa185-85c0-4a75-8fa1-c66ae73156dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=d85f2cc06c80451bb5ced17157ac0271, user_id=8afd12d701ac4fabb3d3f18a6cc76833, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:08.826120 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07aa185-85c0-4a75-8fa1-c66ae73156dd admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:08.837052 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 887/1773] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:21:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 02 19:21:08.943160 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 887/1774] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:21:08 2022] PUT /identity/v3/projects/d85f2cc06c80451bb5ced17157ac0271/users/8afd12d701ac4fabb3d3f18a6cc76833/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:08.952683 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6dbd2e-470b-4e1d-b426-09c58866ea0e None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:08.953806 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6dbd2e-470b-4e1d-b426-09c58866ea0e None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:08.954278 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6dbd2e-470b-4e1d-b426-09c58866ea0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:09.014240 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-9f6dbd2e-470b-4e1d-b426-09c58866ea0e None None] MFA Rules not processed for user `8afd12d701ac4fabb3d3f18a6cc76833`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:09.015725 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-9f6dbd2e-470b-4e1d-b426-09c58866ea0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:09.296975 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 888/1775] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:08 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:21:09.301891 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07aa185-85c0-4a75-8fa1-c66ae73156dd admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:09.436365 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07aa185-85c0-4a75-8fa1-c66ae73156dd admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:09.444682 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b71f31b-5191-42de-8e07-b25d1e14e292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:09.445546 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b71f31b-5191-42de-8e07-b25d1e14e292 admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:09.445927 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b71f31b-5191-42de-8e07-b25d1e14e292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:09.446302 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b71f31b-5191-42de-8e07-b25d1e14e292 admin admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:09.447149 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b71f31b-5191-42de-8e07-b25d1e14e292 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:09.450413 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b71f31b-5191-42de-8e07-b25d1e14e292 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:09.470367 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:09.571974 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 888/1776] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:21:09 2022] POST /identity/v3/projects => generated 367 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:21:09.573082 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6dbd2e-470b-4e1d-b426-09c58866ea0e None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:09.601599 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6dbd2e-470b-4e1d-b426-09c58866ea0e None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:09.760044 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6b8f38-bac7-4cfb-8662-718a69a4f147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:09.760947 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b8f38-bac7-4cfb-8662-718a69a4f147 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:09.762107 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b8f38-bac7-4cfb-8662-718a69a4f147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:09.762741 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b8f38-bac7-4cfb-8662-718a69a4f147 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:09.763851 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6b8f38-bac7-4cfb-8662-718a69a4f147 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:09.766306 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6b8f38-bac7-4cfb-8662-718a69a4f147 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:09.844313 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b71f31b-5191-42de-8e07-b25d1e14e292 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:09.847958 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-6b71f31b-5191-42de-8e07-b25d1e14e292 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:09.889924 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 889/1777] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:21:09 2022] POST /identity/v3/users => generated 290 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:21:09.892979 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b71f31b-5191-42de-8e07-b25d1e14e292 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:09.893928 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6b8f38-bac7-4cfb-8662-718a69a4f147 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:09.916963 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6b8f38-bac7-4cfb-8662-718a69a4f147 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:09.929439 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff197ec-55cc-4fd1-a181-e18ec661abaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:09.932756 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff197ec-55cc-4fd1-a181-e18ec661abaa admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:09.933119 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff197ec-55cc-4fd1-a181-e18ec661abaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:09.933419 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff197ec-55cc-4fd1-a181-e18ec661abaa admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:09.939383 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff197ec-55cc-4fd1-a181-e18ec661abaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:09.944333 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff197ec-55cc-4fd1-a181-e18ec661abaa admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:09.968013 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c01c50-fc05-4182-bf16-ce75f8cd127f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:09.972348 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c01c50-fc05-4182-bf16-ce75f8cd127f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:09.972647 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c01c50-fc05-4182-bf16-ce75f8cd127f admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:09.972938 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c01c50-fc05-4182-bf16-ce75f8cd127f admin admin] PATH_INFO: `/v3/projects/99f1844d6f9b482ab843e01922db5015` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:09.974550 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 890/1778] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:21:09 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 02 19:21:09.978690 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff197ec-55cc-4fd1-a181-e18ec661abaa admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:09.981207 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c01c50-fc05-4182-bf16-ce75f8cd127f admin admin] RBAC: Authorizing `identity:delete_project(project_id=99f1844d6f9b482ab843e01922db5015)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:09.985351 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c01c50-fc05-4182-bf16-ce75f8cd127f admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:09.999151 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff197ec-55cc-4fd1-a181-e18ec661abaa admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:10.024701 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b02267-fd7e-4bcd-981b-4cd8a21c9156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:10.025745 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b02267-fd7e-4bcd-981b-4cd8a21c9156 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:10.026107 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b02267-fd7e-4bcd-981b-4cd8a21c9156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:10.026480 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b02267-fd7e-4bcd-981b-4cd8a21c9156 admin admin] PATH_INFO: `/v3/projects/2b8ab5a2d2444c9788905c96e54422ea/users/e43f779027a5494ab65f7852e95bc328/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:10.069805 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b02267-fd7e-4bcd-981b-4cd8a21c9156 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b8ab5a2d2444c9788905c96e54422ea, user_id=e43f779027a5494ab65f7852e95bc328, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:10.080779 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b02267-fd7e-4bcd-981b-4cd8a21c9156 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:10.085618 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:10.180704 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 891/1779] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:21:09 2022] PUT /identity/v3/projects/2b8ab5a2d2444c9788905c96e54422ea/users/e43f779027a5494ab65f7852e95bc328/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:10.184943 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b02267-fd7e-4bcd-981b-4cd8a21c9156 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:10.263546 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-57c01c50-fc05-4182-bf16-ce75f8cd127f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99f1844d6f9b482ab843e01922db5015', 'request_id': 'req-57c01c50-fc05-4182-bf16-ce75f8cd127f'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:10.324981 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-57c01c50-fc05-4182-bf16-ce75f8cd127f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99f1844d6f9b482ab843e01922db5015', 'request_id': 'req-57c01c50-fc05-4182-bf16-ce75f8cd127f'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:10.328334 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b02267-fd7e-4bcd-981b-4cd8a21c9156 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:10.338846 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-57c01c50-fc05-4182-bf16-ce75f8cd127f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99f1844d6f9b482ab843e01922db5015', 'request_id': 'req-57c01c50-fc05-4182-bf16-ce75f8cd127f'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:10.370466 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-57c01c50-fc05-4182-bf16-ce75f8cd127f admin admin] The token cache is being invalidate because project 99f1844d6f9b482ab843e01922db5015 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:21:10.370778 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-57c01c50-fc05-4182-bf16-ce75f8cd127f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:10.371619 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-57c01c50-fc05-4182-bf16-ce75f8cd127f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:10.378828 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 889/1780] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:21:09 2022] DELETE /identity/v3/projects/99f1844d6f9b482ab843e01922db5015 => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:10.391704 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c01c50-fc05-4182-bf16-ce75f8cd127f admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:10.399405 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-57c01c50-fc05-4182-bf16-ce75f8cd127f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:10.475879 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0adc745-d7f7-40b9-b12b-50e23d49d1d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:10.476750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0adc745-d7f7-40b9-b12b-50e23d49d1d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:10.477091 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0adc745-d7f7-40b9-b12b-50e23d49d1d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:10.477318 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0adc745-d7f7-40b9-b12b-50e23d49d1d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:10.478114 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0adc745-d7f7-40b9-b12b-50e23d49d1d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:10.479964 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0adc745-d7f7-40b9-b12b-50e23d49d1d9 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:10.511546 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 892/1781] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:21:10 2022] GET /identity/v3/roles => generated 1420 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:21:10.599672 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c01c50-fc05-4182-bf16-ce75f8cd127f admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:10.599672 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c351afa-a802-452b-b0e2-3f2e30c7f7b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:10.599672 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c351afa-a802-452b-b0e2-3f2e30c7f7b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:10.599672 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c351afa-a802-452b-b0e2-3f2e30c7f7b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:10.601217 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c351afa-a802-452b-b0e2-3f2e30c7f7b2 admin admin] PATH_INFO: `/v3/users/21cb2a1781e9442d9078070fea1185aa` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:10.601217 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c351afa-a802-452b-b0e2-3f2e30c7f7b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21cb2a1781e9442d9078070fea1185aa)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:10.601217 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c351afa-a802-452b-b0e2-3f2e30c7f7b2 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:10.602248 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0adc745-d7f7-40b9-b12b-50e23d49d1d9 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:10.604622 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-f0adc745-d7f7-40b9-b12b-50e23d49d1d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:10.644299 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0adc745-d7f7-40b9-b12b-50e23d49d1d9 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:10.649641 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca08dd25-7dd9-4814-9167-320a9b4e47ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:10.650611 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca08dd25-7dd9-4814-9167-320a9b4e47ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:10.650954 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca08dd25-7dd9-4814-9167-320a9b4e47ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:10.651829 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca08dd25-7dd9-4814-9167-320a9b4e47ad admin admin] PATH_INFO: `/v3/projects/2b8ab5a2d2444c9788905c96e54422ea/users/e43f779027a5494ab65f7852e95bc328/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:10.665230 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca08dd25-7dd9-4814-9167-320a9b4e47ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b8ab5a2d2444c9788905c96e54422ea, user_id=e43f779027a5494ab65f7852e95bc328, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:10.669090 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca08dd25-7dd9-4814-9167-320a9b4e47ad admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:10.715907 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 893/1782] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:21:10 2022] PUT /identity/v3/projects/2b8ab5a2d2444c9788905c96e54422ea/users/e43f779027a5494ab65f7852e95bc328/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:10.717486 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a3d93-76b2-4172-8ca7-ef5102ec6665 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:10.717804 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a3d93-76b2-4172-8ca7-ef5102ec6665 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:10.718258 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a3d93-76b2-4172-8ca7-ef5102ec6665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:10.803992 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-1c4a3d93-76b2-4172-8ca7-ef5102ec6665 None None] MFA Rules not processed for user `e43f779027a5494ab65f7852e95bc328`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:10.806860 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-1c4a3d93-76b2-4172-8ca7-ef5102ec6665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:10.909183 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 894/1783] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:10 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:21:10.923590 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4a3d93-76b2-4172-8ca7-ef5102ec6665 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:10.926491 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-1c4a3d93-76b2-4172-8ca7-ef5102ec6665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:10.946359 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:11.013045 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4a3d93-76b2-4172-8ca7-ef5102ec6665 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:11.018732 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d93996d-e49b-49b2-899b-60442807fc70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:11.019507 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93996d-e49b-49b2-899b-60442807fc70 service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:11.019796 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93996d-e49b-49b2-899b-60442807fc70 service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:11.020161 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93996d-e49b-49b2-899b-60442807fc70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:11.032752 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d93996d-e49b-49b2-899b-60442807fc70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:11.035536 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d93996d-e49b-49b2-899b-60442807fc70 service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:11.049067 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 895/1784] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:21:10 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 02 19:21:11.930865 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d93996d-e49b-49b2-899b-60442807fc70 service cinder] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:11.932485 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-2d93996d-e49b-49b2-899b-60442807fc70 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:11.962348 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d93996d-e49b-49b2-899b-60442807fc70 service cinder] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:11.976396 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac826c29-4c10-4a71-afce-910e80a15804 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:11.978008 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac826c29-4c10-4a71-afce-910e80a15804 service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:11.979233 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac826c29-4c10-4a71-afce-910e80a15804 service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:11.979766 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac826c29-4c10-4a71-afce-910e80a15804 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:11.985604 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-ac826c29-4c10-4a71-afce-910e80a15804 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:12.044676 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-9c351afa-a802-452b-b0e2-3f2e30c7f7b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21cb2a1781e9442d9078070fea1185aa'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:12.061863 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-9c351afa-a802-452b-b0e2-3f2e30c7f7b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21cb2a1781e9442d9078070fea1185aa'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:12.063456 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-9c351afa-a802-452b-b0e2-3f2e30c7f7b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21cb2a1781e9442d9078070fea1185aa'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:12.117643 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac826c29-4c10-4a71-afce-910e80a15804 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:12.119729 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac826c29-4c10-4a71-afce-910e80a15804 service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:12.134706 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 896/1785] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:21:11 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 02 19:21:12.173573 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-9c351afa-a802-452b-b0e2-3f2e30c7f7b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21cb2a1781e9442d9078070fea1185aa'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:12.177191 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-9c351afa-a802-452b-b0e2-3f2e30c7f7b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21cb2a1781e9442d9078070fea1185aa'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:12.196200 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7d0b5-dbd9-4b60-8f55-7be32416707a None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:12.196624 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7d0b5-dbd9-4b60-8f55-7be32416707a None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:12.197001 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7d0b5-dbd9-4b60-8f55-7be32416707a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:12.219415 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=81,keystone:INSERT=3,keystone:DELETE=16 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:12.245914 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 890/1786] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:21:10 2022] DELETE /identity/v3/users/21cb2a1781e9442d9078070fea1185aa => generated 0 bytes in 1855 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:12.258599 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-aac7d0b5-dbd9-4b60-8f55-7be32416707a None None] MFA Rules not processed for user `a0681f3eff324262b4289bf6d8f4c311`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:12.263873 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-aac7d0b5-dbd9-4b60-8f55-7be32416707a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:12.433490 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 897/1787] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:21:12.995752 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d2eb4-7883-4c3e-b861-09f86a8f1598 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:12.996164 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d2eb4-7883-4c3e-b861-09f86a8f1598 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:12.996563 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d2eb4-7883-4c3e-b861-09f86a8f1598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:13.038338 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-df1d2eb4-7883-4c3e-b861-09f86a8f1598 None None] MFA Rules not processed for user `df62e9ab4d044dcaab00c5d76416e3e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:13.038338 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-df1d2eb4-7883-4c3e-b861-09f86a8f1598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:13.155451 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 891/1788] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:12 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:21:13.234477 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9ff998-1e91-4037-9194-87a3195eb3d2 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:13.236504 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9ff998-1e91-4037-9194-87a3195eb3d2 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:13.236504 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9ff998-1e91-4037-9194-87a3195eb3d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:13.268209 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-ab9ff998-1e91-4037-9194-87a3195eb3d2 None None] MFA Rules not processed for user `72558099db31474292377bab47035911`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:13.269203 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-ab9ff998-1e91-4037-9194-87a3195eb3d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:13.353710 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:13.397570 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 898/1789] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:13 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:21:13.402109 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1d2eb4-7883-4c3e-b861-09f86a8f1598 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:13.404706 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-df1d2eb4-7883-4c3e-b861-09f86a8f1598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:13.487389 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9ff998-1e91-4037-9194-87a3195eb3d2 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:13.489451 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-ab9ff998-1e91-4037-9194-87a3195eb3d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:13.533468 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1d2eb4-7883-4c3e-b861-09f86a8f1598 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:13.538711 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51328bae-5da2-47fd-aa28-dbf841529ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:13.539430 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51328bae-5da2-47fd-aa28-dbf841529ce0 admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:13.539720 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51328bae-5da2-47fd-aa28-dbf841529ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:13.540005 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51328bae-5da2-47fd-aa28-dbf841529ce0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:13.540735 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51328bae-5da2-47fd-aa28-dbf841529ce0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:13.542698 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51328bae-5da2-47fd-aa28-dbf841529ce0 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:13.616245 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9ff998-1e91-4037-9194-87a3195eb3d2 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:13.621383 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75875019-b294-4558-915e-e7b8d9dc2fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:13.622425 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75875019-b294-4558-915e-e7b8d9dc2fd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:13.622852 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75875019-b294-4558-915e-e7b8d9dc2fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:13.623239 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75875019-b294-4558-915e-e7b8d9dc2fd9 admin admin] PATH_INFO: `/v3/projects/801a3fb14c9f49fda0e305d00253bb95` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:13.624774 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75875019-b294-4558-915e-e7b8d9dc2fd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=801a3fb14c9f49fda0e305d00253bb95)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:13.628876 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75875019-b294-4558-915e-e7b8d9dc2fd9 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:13.755711 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 892/1790] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:21:13 2022] POST /identity/v3/projects => generated 369 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 02 19:21:13.756876 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51328bae-5da2-47fd-aa28-dbf841529ce0 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:13.793807 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-75875019-b294-4558-915e-e7b8d9dc2fd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '801a3fb14c9f49fda0e305d00253bb95', 'request_id': 'req-75875019-b294-4558-915e-e7b8d9dc2fd9'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:13.865296 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51328bae-5da2-47fd-aa28-dbf841529ce0 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:13.879409 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-75875019-b294-4558-915e-e7b8d9dc2fd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '801a3fb14c9f49fda0e305d00253bb95', 'request_id': 'req-75875019-b294-4558-915e-e7b8d9dc2fd9'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:13.886259 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-75875019-b294-4558-915e-e7b8d9dc2fd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '801a3fb14c9f49fda0e305d00253bb95', 'request_id': 'req-75875019-b294-4558-915e-e7b8d9dc2fd9'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:13.908399 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-75875019-b294-4558-915e-e7b8d9dc2fd9 admin admin] The token cache is being invalidate because project 801a3fb14c9f49fda0e305d00253bb95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:21:13.910348 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-75875019-b294-4558-915e-e7b8d9dc2fd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:13.912304 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-75875019-b294-4558-915e-e7b8d9dc2fd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:13.933720 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 899/1791] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:21:13 2022] DELETE /identity/v3/projects/801a3fb14c9f49fda0e305d00253bb95 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 02 19:21:13.935054 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c8449-91c8-450e-a3b6-d15f403b8496 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:13.935450 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c8449-91c8-450e-a3b6-d15f403b8496 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:13.935868 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c8449-91c8-450e-a3b6-d15f403b8496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:13.959551 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-414c8449-91c8-450e-a3b6-d15f403b8496 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:13.960923 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-414c8449-91c8-450e-a3b6-d15f403b8496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:13.968623 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4852a524-25b3-4fcf-b575-a7c88f28cce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:13.969586 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4852a524-25b3-4fcf-b575-a7c88f28cce9 admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:13.970170 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4852a524-25b3-4fcf-b575-a7c88f28cce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:13.970682 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4852a524-25b3-4fcf-b575-a7c88f28cce9 admin admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:13.971608 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4852a524-25b3-4fcf-b575-a7c88f28cce9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:13.974289 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4852a524-25b3-4fcf-b575-a7c88f28cce9 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:13.974788 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 900/1792] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:21:13.991379 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8d7111-0112-4442-a8b6-798c07b74266 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:13.992106 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8d7111-0112-4442-a8b6-798c07b74266 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:13.993506 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8d7111-0112-4442-a8b6-798c07b74266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:14.025330 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-2e8d7111-0112-4442-a8b6-798c07b74266 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:14.026783 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-2e8d7111-0112-4442-a8b6-798c07b74266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:14.044228 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 901/1793] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:21:14.045919 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8d7111-0112-4442-a8b6-798c07b74266 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:14.061504 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8d7111-0112-4442-a8b6-798c07b74266 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:14.067619 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ea2dba-241d-4424-990b-bc493fd561e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:14.068873 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea2dba-241d-4424-990b-bc493fd561e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:14.071124 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea2dba-241d-4424-990b-bc493fd561e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:14.071636 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea2dba-241d-4424-990b-bc493fd561e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:14.072566 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ea2dba-241d-4424-990b-bc493fd561e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:21:14.073021 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ea2dba-241d-4424-990b-bc493fd561e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:14.075060 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ea2dba-241d-4424-990b-bc493fd561e1 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:14.090044 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 902/1794] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 2 19:21:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:21:14.092355 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c8e56-67b1-451f-befe-e1d74a120e87 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:14.092726 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c8e56-67b1-451f-befe-e1d74a120e87 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:14.093208 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c8e56-67b1-451f-befe-e1d74a120e87 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:14.095331 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 893/1795] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:21:13 2022] POST /identity/v3/users => generated 291 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:21:14.099033 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 903/1796] 10.222.0.33 () {54 vars in 912 bytes} [Thu Jun 2 19:21:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:21:14.099926 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6c8e56-67b1-451f-befe-e1d74a120e87 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:14.100442 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4852a524-25b3-4fcf-b575-a7c88f28cce9 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:14.102370 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-ca6c8e56-67b1-451f-befe-e1d74a120e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:14.125768 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4852a524-25b3-4fcf-b575-a7c88f28cce9 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:14.125903 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6c8e56-67b1-451f-befe-e1d74a120e87 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:14.136535 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff90420-e468-46b5-9a77-45719dca5f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:14.137228 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff90420-e468-46b5-9a77-45719dca5f0b admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:14.137516 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff90420-e468-46b5-9a77-45719dca5f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:14.137800 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff90420-e468-46b5-9a77-45719dca5f0b admin admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:14.138503 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff90420-e468-46b5-9a77-45719dca5f0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:14.140676 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff90420-e468-46b5-9a77-45719dca5f0b admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:14.143093 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373c2c3f-4108-4d9a-8882-5c509629d828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:14.143811 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c2c3f-4108-4d9a-8882-5c509629d828 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:14.144114 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c2c3f-4108-4d9a-8882-5c509629d828 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:14.144388 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c2c3f-4108-4d9a-8882-5c509629d828 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:14.145128 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373c2c3f-4108-4d9a-8882-5c509629d828 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:14.147052 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373c2c3f-4108-4d9a-8882-5c509629d828 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:14.173675 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 904/1797] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:21:14 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:21:14.175611 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ba5b6-3cb3-4ff5-b32d-2ab31e0e638d None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:14.176003 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ba5b6-3cb3-4ff5-b32d-2ab31e0e638d None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:14.176297 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ba5b6-3cb3-4ff5-b32d-2ab31e0e638d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:14.196114 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-8a1ba5b6-3cb3-4ff5-b32d-2ab31e0e638d None None] MFA Rules not processed for user `7417a54046e34e899d7b03f2f2314deb`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:14.200181 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-8a1ba5b6-3cb3-4ff5-b32d-2ab31e0e638d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:14.242719 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:14.270988 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 905/1798] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Jun 2 19:21:14 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:21:14.277507 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1ba5b6-3cb3-4ff5-b32d-2ab31e0e638d None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:14.290852 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1ba5b6-3cb3-4ff5-b32d-2ab31e0e638d None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:14.296447 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b03f1d-e26c-4367-b132-d12a0eaf8038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:14.297717 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b03f1d-e26c-4367-b132-d12a0eaf8038 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:14.297834 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b03f1d-e26c-4367-b132-d12a0eaf8038 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:14.298308 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b03f1d-e26c-4367-b132-d12a0eaf8038 admin admin] PATH_INFO: `/v3/projects/7118cfff01c34c8abb4aa2144013c485/users/b5337f7eeef24e34a79ef8c6ba4d027e/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:14.308821 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 894/1799] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:21:14 2022] POST /identity/v3/projects => generated 365 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:21:14.308821 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff90420-e468-46b5-9a77-45719dca5f0b admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:14.316485 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-0ff90420-e468-46b5-9a77-45719dca5f0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:14.367579 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b03f1d-e26c-4367-b132-d12a0eaf8038 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7118cfff01c34c8abb4aa2144013c485, user_id=b5337f7eeef24e34a79ef8c6ba4d027e, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:14.369184 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b03f1d-e26c-4367-b132-d12a0eaf8038 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:14.419345 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff90420-e468-46b5-9a77-45719dca5f0b admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:14.427138 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac2c81e-f08b-437c-9993-d2fb2fe052e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:14.427824 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac2c81e-f08b-437c-9993-d2fb2fe052e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:14.428116 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac2c81e-f08b-437c-9993-d2fb2fe052e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:14.428402 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac2c81e-f08b-437c-9993-d2fb2fe052e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:14.446048 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac2c81e-f08b-437c-9993-d2fb2fe052e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:14.457910 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 906/1800] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:21:14 2022] PUT /identity/v3/projects/7118cfff01c34c8abb4aa2144013c485/users/b5337f7eeef24e34a79ef8c6ba4d027e/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:14.461690 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b03f1d-e26c-4367-b132-d12a0eaf8038 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:14.465676 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac2c81e-f08b-437c-9993-d2fb2fe052e0 service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:14.520972 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 895/1801] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:21:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:21:14.520972 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac2c81e-f08b-437c-9993-d2fb2fe052e0 service neutron] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:14.651750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b03f1d-e26c-4367-b132-d12a0eaf8038 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:14.659843 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7835b528-4669-4de4-b889-b2a7f4f3eae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:14.660791 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7835b528-4669-4de4-b889-b2a7f4f3eae6 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:14.661236 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7835b528-4669-4de4-b889-b2a7f4f3eae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:14.661663 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7835b528-4669-4de4-b889-b2a7f4f3eae6 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:14.662720 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7835b528-4669-4de4-b889-b2a7f4f3eae6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:14.665508 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7835b528-4669-4de4-b889-b2a7f4f3eae6 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:14.665690 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac2c81e-f08b-437c-9993-d2fb2fe052e0 service neutron] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:14.692477 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f265a25-e400-4f2e-b1d4-ded0d78bd91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:14.693182 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f265a25-e400-4f2e-b1d4-ded0d78bd91e admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:14.693473 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f265a25-e400-4f2e-b1d4-ded0d78bd91e admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:14.693755 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f265a25-e400-4f2e-b1d4-ded0d78bd91e admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:14.699342 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f265a25-e400-4f2e-b1d4-ded0d78bd91e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:14.703507 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f265a25-e400-4f2e-b1d4-ded0d78bd91e admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:14.754113 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 896/1802] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:21:14 2022] GET /identity/v3/roles => generated 1420 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:21:14.761711 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e380727e-57d2-414d-9871-58ade60cc593 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:14.762096 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e380727e-57d2-414d-9871-58ade60cc593 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:14.762293 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e380727e-57d2-414d-9871-58ade60cc593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:14.775629 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 907/1803] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:21:14 2022] POST /identity/v3/users => generated 283 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:21:14.776679 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7835b528-4669-4de4-b889-b2a7f4f3eae6 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:14.803201 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7835b528-4669-4de4-b889-b2a7f4f3eae6 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:14.818108 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e7ce8f-e7c6-4f7d-957d-a8f0a5aea2e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:14.821186 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e7ce8f-e7c6-4f7d-957d-a8f0a5aea2e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:14.821476 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e7ce8f-e7c6-4f7d-957d-a8f0a5aea2e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:14.823581 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e7ce8f-e7c6-4f7d-957d-a8f0a5aea2e8 admin admin] PATH_INFO: `/v3/projects/7118cfff01c34c8abb4aa2144013c485/users/b5337f7eeef24e34a79ef8c6ba4d027e/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:14.832080 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e7ce8f-e7c6-4f7d-957d-a8f0a5aea2e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7118cfff01c34c8abb4aa2144013c485, user_id=b5337f7eeef24e34a79ef8c6ba4d027e, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:14.838877 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e7ce8f-e7c6-4f7d-957d-a8f0a5aea2e8 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:14.857650 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-e380727e-57d2-414d-9871-58ade60cc593 None None] MFA Rules not processed for user `35d9904f560f455dbcc07ef0436d1a69`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:14.858967 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-e380727e-57d2-414d-9871-58ade60cc593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:14.869190 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 908/1804] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:21:14 2022] PUT /identity/v3/projects/7118cfff01c34c8abb4aa2144013c485/users/b5337f7eeef24e34a79ef8c6ba4d027e/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:14.870365 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e7ce8f-e7c6-4f7d-957d-a8f0a5aea2e8 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:14.906511 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:14.980149 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 897/1805] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:14 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:21:14.981739 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e380727e-57d2-414d-9871-58ade60cc593 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:14.983930 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-e380727e-57d2-414d-9871-58ade60cc593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:15.020390 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e7ce8f-e7c6-4f7d-957d-a8f0a5aea2e8 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:15.026448 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f05ce27-7d7d-4916-94eb-e14513266864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:15.028132 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f05ce27-7d7d-4916-94eb-e14513266864 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:15.028434 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f05ce27-7d7d-4916-94eb-e14513266864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:15.028720 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f05ce27-7d7d-4916-94eb-e14513266864 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:15.029623 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f05ce27-7d7d-4916-94eb-e14513266864 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:15.031593 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f05ce27-7d7d-4916-94eb-e14513266864 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:15.052356 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 909/1806] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:21:14 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:21:15.056151 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f05ce27-7d7d-4916-94eb-e14513266864 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:15.092760 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f05ce27-7d7d-4916-94eb-e14513266864 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:15.098190 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e380727e-57d2-414d-9871-58ade60cc593 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:15.104572 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e80c104-1793-4fe7-aeb1-54d791307179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:15.105273 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e80c104-1793-4fe7-aeb1-54d791307179 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:15.105566 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e80c104-1793-4fe7-aeb1-54d791307179 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:15.106129 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e80c104-1793-4fe7-aeb1-54d791307179 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:15.112182 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e80c104-1793-4fe7-aeb1-54d791307179 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:15.113629 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e80c104-1793-4fe7-aeb1-54d791307179 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:15.123698 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf1fd5-97fa-4d80-b3b6-1da87320d645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:15.126688 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf1fd5-97fa-4d80-b3b6-1da87320d645 admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:15.126688 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf1fd5-97fa-4d80-b3b6-1da87320d645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:15.126688 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf1fd5-97fa-4d80-b3b6-1da87320d645 admin admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:15.126688 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cf1fd5-97fa-4d80-b3b6-1da87320d645 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:15.133105 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cf1fd5-97fa-4d80-b3b6-1da87320d645 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:15.149978 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 910/1807] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:21:15 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 02 19:21:15.152795 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e80c104-1793-4fe7-aeb1-54d791307179 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:15.179051 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e80c104-1793-4fe7-aeb1-54d791307179 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:15.194010 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0494e87-1403-4fc4-99e8-1a39510694b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:15.194984 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0494e87-1403-4fc4-99e8-1a39510694b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:15.197545 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0494e87-1403-4fc4-99e8-1a39510694b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:15.197545 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0494e87-1403-4fc4-99e8-1a39510694b2 admin admin] PATH_INFO: `/v3/projects/0a3fac8d71564ccc8639a0c9fc15f563/users/8160092e2ab84839ba807f746fffc570/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:15.227283 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0494e87-1403-4fc4-99e8-1a39510694b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a3fac8d71564ccc8639a0c9fc15f563, user_id=8160092e2ab84839ba807f746fffc570, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:15.230906 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0494e87-1403-4fc4-99e8-1a39510694b2 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:15.282450 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 898/1808] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:21:14 2022] POST /identity/v3/projects => generated 403 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:21:15.286173 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf1fd5-97fa-4d80-b3b6-1da87320d645 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:15.349656 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 911/1809] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:21:15 2022] PUT /identity/v3/projects/0a3fac8d71564ccc8639a0c9fc15f563/users/8160092e2ab84839ba807f746fffc570/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:15.375624 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c8d2a7-5a36-440a-b330-047f9683e3bc None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:15.375624 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c8d2a7-5a36-440a-b330-047f9683e3bc None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:15.375624 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c8d2a7-5a36-440a-b330-047f9683e3bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:15.376413 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf1fd5-97fa-4d80-b3b6-1da87320d645 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:15.485461 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451fcc85-ef77-4a54-9384-d500abbb78d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:15.486280 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-c8c8d2a7-5a36-440a-b330-047f9683e3bc None None] MFA Rules not processed for user `8160092e2ab84839ba807f746fffc570`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:15.486434 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451fcc85-ef77-4a54-9384-d500abbb78d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:15.486700 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451fcc85-ef77-4a54-9384-d500abbb78d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:15.487795 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451fcc85-ef77-4a54-9384-d500abbb78d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:15.488508 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451fcc85-ef77-4a54-9384-d500abbb78d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:15.489264 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-c8c8d2a7-5a36-440a-b330-047f9683e3bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:15.490544 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451fcc85-ef77-4a54-9384-d500abbb78d4 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:15.619622 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 899/1810] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:21:15 2022] POST /identity/v3/users => generated 308 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 02 19:21:15.621133 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451fcc85-ef77-4a54-9384-d500abbb78d4 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:15.643588 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451fcc85-ef77-4a54-9384-d500abbb78d4 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:15.674465 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990562c7-6b3b-4386-a4d1-b74cf34fed75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:15.674465 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990562c7-6b3b-4386-a4d1-b74cf34fed75 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:15.674465 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990562c7-6b3b-4386-a4d1-b74cf34fed75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:15.674465 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990562c7-6b3b-4386-a4d1-b74cf34fed75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:15.675638 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990562c7-6b3b-4386-a4d1-b74cf34fed75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:15.677362 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990562c7-6b3b-4386-a4d1-b74cf34fed75 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:15.739061 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 900/1811] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:21:15 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 02 19:21:15.743680 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990562c7-6b3b-4386-a4d1-b74cf34fed75 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:15.754531 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 912/1812] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:15 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 392 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:21:15.767995 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c8d2a7-5a36-440a-b330-047f9683e3bc None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:15.780488 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c8d2a7-5a36-440a-b330-047f9683e3bc None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:15.784709 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990562c7-6b3b-4386-a4d1-b74cf34fed75 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:15.788716 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0886258-fd42-4d75-b99a-88e7c487dfa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:15.792751 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0886258-fd42-4d75-b99a-88e7c487dfa9 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:15.792751 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0886258-fd42-4d75-b99a-88e7c487dfa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:15.792751 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0886258-fd42-4d75-b99a-88e7c487dfa9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:15.792751 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0886258-fd42-4d75-b99a-88e7c487dfa9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:15.798038 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0886258-fd42-4d75-b99a-88e7c487dfa9 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:15.803952 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47183df2-9251-4ffc-a01e-cd435ac7ea13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:15.805153 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47183df2-9251-4ffc-a01e-cd435ac7ea13 admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:15.807434 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47183df2-9251-4ffc-a01e-cd435ac7ea13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:15.807908 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47183df2-9251-4ffc-a01e-cd435ac7ea13 admin admin] PATH_INFO: `/v3/projects/fc9a666a9e7d411fb53b5571866670e7/users/e6fe13ff783f4171957762a1daff2f52/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:15.878916 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47183df2-9251-4ffc-a01e-cd435ac7ea13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc9a666a9e7d411fb53b5571866670e7, user_id=e6fe13ff783f4171957762a1daff2f52, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:15.881519 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47183df2-9251-4ffc-a01e-cd435ac7ea13 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:15.917476 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 913/1813] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:21:15 2022] POST /identity/v3/projects => generated 363 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 02 19:21:15.918916 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0855c3fb-5bd5-46a1-8a8c-d812a1e21436 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:15.919215 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0855c3fb-5bd5-46a1-8a8c-d812a1e21436 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:15.919496 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0855c3fb-5bd5-46a1-8a8c-d812a1e21436 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:15.924307 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 914/1814] 10.222.0.33 () {54 vars in 921 bytes} [Thu Jun 2 19:21:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:21:15.925777 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0855c3fb-5bd5-46a1-8a8c-d812a1e21436 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:16.010468 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 901/1815] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:21:15 2022] PUT /identity/v3/projects/fc9a666a9e7d411fb53b5571866670e7/users/e6fe13ff783f4171957762a1daff2f52/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:16.010468 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8483d49-3fc4-4722-bd3b-e5b58fb3175d None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:16.010468 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8483d49-3fc4-4722-bd3b-e5b58fb3175d None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:16.010468 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8483d49-3fc4-4722-bd3b-e5b58fb3175d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:16.055693 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-d8483d49-3fc4-4722-bd3b-e5b58fb3175d None None] MFA Rules not processed for user `70664d5cbdee47ee89c120761939cc75`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:16.056970 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-d8483d49-3fc4-4722-bd3b-e5b58fb3175d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:16.083114 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0855c3fb-5bd5-46a1-8a8c-d812a1e21436 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:16.230814 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817a008d-f6be-48f6-93e0-338f88a4fbd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:16.240004 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a008d-f6be-48f6-93e0-338f88a4fbd5 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:16.241890 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a008d-f6be-48f6-93e0-338f88a4fbd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:16.242184 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a008d-f6be-48f6-93e0-338f88a4fbd5 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:16.243009 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817a008d-f6be-48f6-93e0-338f88a4fbd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:16.246061 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 902/1816] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Jun 2 19:21:16 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:21:16.247884 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8483d49-3fc4-4722-bd3b-e5b58fb3175d None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:16.248261 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817a008d-f6be-48f6-93e0-338f88a4fbd5 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:16.268943 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8483d49-3fc4-4722-bd3b-e5b58fb3175d None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:16.291792 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a0dd1-b096-4c75-b10d-cfc86a80d030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:16.292639 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a0dd1-b096-4c75-b10d-cfc86a80d030 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:16.294171 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a0dd1-b096-4c75-b10d-cfc86a80d030 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:16.294476 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a0dd1-b096-4c75-b10d-cfc86a80d030 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:16.296800 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9a0dd1-b096-4c75-b10d-cfc86a80d030 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:16.298526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9a0dd1-b096-4c75-b10d-cfc86a80d030 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:16.313412 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 903/1817] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:21:16 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:21:16.314631 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a0dd1-b096-4c75-b10d-cfc86a80d030 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:16.327353 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a0dd1-b096-4c75-b10d-cfc86a80d030 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:16.338470 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3798708a-7449-4391-96c4-3288b28886c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:16.338470 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3798708a-7449-4391-96c4-3288b28886c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:16.338470 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3798708a-7449-4391-96c4-3288b28886c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:16.338470 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3798708a-7449-4391-96c4-3288b28886c0 admin admin] PATH_INFO: `/v3/projects/fc9a666a9e7d411fb53b5571866670e7/users/e6fe13ff783f4171957762a1daff2f52/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:16.338952 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3798708a-7449-4391-96c4-3288b28886c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc9a666a9e7d411fb53b5571866670e7, user_id=e6fe13ff783f4171957762a1daff2f52, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:16.339487 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3798708a-7449-4391-96c4-3288b28886c0 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:16.351776 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 904/1818] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:21:16 2022] PUT /identity/v3/projects/fc9a666a9e7d411fb53b5571866670e7/users/e6fe13ff783f4171957762a1daff2f52/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:16.397649 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3798708a-7449-4391-96c4-3288b28886c0 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:16.401711 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-3798708a-7449-4391-96c4-3288b28886c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:16.416983 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 915/1819] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:21:15 2022] POST /identity/v3/users => generated 288 bytes in 492 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:21:16.420138 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817a008d-f6be-48f6-93e0-338f88a4fbd5 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:16.452257 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817a008d-f6be-48f6-93e0-338f88a4fbd5 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:16.547747 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3798708a-7449-4391-96c4-3288b28886c0 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:16.565712 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe9928b-696b-41f6-a4d1-30d4252c98db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:16.566568 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe9928b-696b-41f6-a4d1-30d4252c98db service nova] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:16.567065 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe9928b-696b-41f6-a4d1-30d4252c98db service nova] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:16.567517 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe9928b-696b-41f6-a4d1-30d4252c98db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:16.586709 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50a88a6-5b23-4cac-9fff-e5d9c8717903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:16.587694 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50a88a6-5b23-4cac-9fff-e5d9c8717903 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:16.588057 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50a88a6-5b23-4cac-9fff-e5d9c8717903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:16.588448 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50a88a6-5b23-4cac-9fff-e5d9c8717903 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:16.588648 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe9928b-696b-41f6-a4d1-30d4252c98db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:16.589965 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50a88a6-5b23-4cac-9fff-e5d9c8717903 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:16.590600 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe9928b-696b-41f6-a4d1-30d4252c98db service nova] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:16.592389 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50a88a6-5b23-4cac-9fff-e5d9c8717903 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:16.609745 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 905/1820] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:21:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:21:16.617918 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 916/1821] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:21:16 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:21:16.622549 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe9928b-696b-41f6-a4d1-30d4252c98db service nova] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:16.633821 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe9928b-696b-41f6-a4d1-30d4252c98db service nova] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:16.640050 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eb86f6-058a-45ce-9c46-ad3cb3d24415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:16.640987 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb86f6-058a-45ce-9c46-ad3cb3d24415 admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:16.641651 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb86f6-058a-45ce-9c46-ad3cb3d24415 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:16.641651 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb86f6-058a-45ce-9c46-ad3cb3d24415 admin admin] PATH_INFO: `/v3/projects/bec7fef66b7a4145acac655f4d8bbab4/users/1d001e561e7f4be0bae2d0b2878b613b/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:16.674366 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eb86f6-058a-45ce-9c46-ad3cb3d24415 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bec7fef66b7a4145acac655f4d8bbab4, user_id=1d001e561e7f4be0bae2d0b2878b613b, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:16.680559 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eb86f6-058a-45ce-9c46-ad3cb3d24415 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:16.822059 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 906/1822] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:21:16 2022] PUT /identity/v3/projects/bec7fef66b7a4145acac655f4d8bbab4/users/1d001e561e7f4be0bae2d0b2878b613b/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:16.825497 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50a88a6-5b23-4cac-9fff-e5d9c8717903 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:16.845916 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50a88a6-5b23-4cac-9fff-e5d9c8717903 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:16.964703 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec95a89f-99cf-4626-9df4-fe139d19d33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:16.965468 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec95a89f-99cf-4626-9df4-fe139d19d33a admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:16.965762 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec95a89f-99cf-4626-9df4-fe139d19d33a admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:16.967137 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec95a89f-99cf-4626-9df4-fe139d19d33a admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:16.967137 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec95a89f-99cf-4626-9df4-fe139d19d33a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:16.968795 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec95a89f-99cf-4626-9df4-fe139d19d33a admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:16.981720 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 917/1823] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:21:16 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:21:16.985608 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eb86f6-058a-45ce-9c46-ad3cb3d24415 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:17.004660 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eb86f6-058a-45ce-9c46-ad3cb3d24415 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:17.019109 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8fea53-62dd-4dbe-a015-4523b0000669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:17.020018 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8fea53-62dd-4dbe-a015-4523b0000669 admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:17.020535 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8fea53-62dd-4dbe-a015-4523b0000669 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:17.020957 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8fea53-62dd-4dbe-a015-4523b0000669 admin admin] PATH_INFO: `/v3/projects/bec7fef66b7a4145acac655f4d8bbab4/users/1d001e561e7f4be0bae2d0b2878b613b/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:17.025463 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8fea53-62dd-4dbe-a015-4523b0000669 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bec7fef66b7a4145acac655f4d8bbab4, user_id=1d001e561e7f4be0bae2d0b2878b613b, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:17.029818 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8fea53-62dd-4dbe-a015-4523b0000669 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:17.043972 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 907/1824] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:21:16 2022] PUT /identity/v3/projects/bec7fef66b7a4145acac655f4d8bbab4/users/1d001e561e7f4be0bae2d0b2878b613b/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:17.055407 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e53b6-e92c-4716-b51a-adc525faff4a None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:17.064607 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e53b6-e92c-4716-b51a-adc525faff4a None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:17.064607 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e53b6-e92c-4716-b51a-adc525faff4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:17.095065 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-136e53b6-e92c-4716-b51a-adc525faff4a None None] MFA Rules not processed for user `1d001e561e7f4be0bae2d0b2878b613b`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:17.097537 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-136e53b6-e92c-4716-b51a-adc525faff4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:17.250341 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 918/1825] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:17 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:21:17.251274 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8fea53-62dd-4dbe-a015-4523b0000669 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:17.260764 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-9d8fea53-62dd-4dbe-a015-4523b0000669 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:17.335478 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8fea53-62dd-4dbe-a015-4523b0000669 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:17.340070 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37429e05-eb8a-4122-9984-21175aa385c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:17.341096 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37429e05-eb8a-4122-9984-21175aa385c5 service cinder] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:17.341530 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37429e05-eb8a-4122-9984-21175aa385c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:17.342016 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37429e05-eb8a-4122-9984-21175aa385c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:17.353585 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37429e05-eb8a-4122-9984-21175aa385c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:17.355673 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37429e05-eb8a-4122-9984-21175aa385c5 service cinder] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:17.369546 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 908/1826] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:21:17 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 02 19:21:17.417334 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b105cf-7147-4f54-b1b8-4911f16c28bf None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:17.419360 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b105cf-7147-4f54-b1b8-4911f16c28bf None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:17.420542 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b105cf-7147-4f54-b1b8-4911f16c28bf None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:17.422792 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 919/1827] 10.222.0.33 () {54 vars in 921 bytes} [Thu Jun 2 19:21:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:21:17.473937 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8673ab3-c1a4-4af1-b15b-84ec38d9790e None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:17.474481 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8673ab3-c1a4-4af1-b15b-84ec38d9790e None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:17.474481 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8673ab3-c1a4-4af1-b15b-84ec38d9790e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:17.509206 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-f8673ab3-c1a4-4af1-b15b-84ec38d9790e None None] MFA Rules not processed for user `70664d5cbdee47ee89c120761939cc75`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:17.513946 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-f8673ab3-c1a4-4af1-b15b-84ec38d9790e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:17.554969 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:17.666958 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 909/1828] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Jun 2 19:21:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:21:17.761234 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b105cf-7147-4f54-b1b8-4911f16c28bf None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:17.769098 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-25b105cf-7147-4f54-b1b8-4911f16c28bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:17.779715 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8673ab3-c1a4-4af1-b15b-84ec38d9790e None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:17.792564 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8673ab3-c1a4-4af1-b15b-84ec38d9790e None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:17.800325 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b105cf-7147-4f54-b1b8-4911f16c28bf None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:17.805102 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dca5478-fe19-4db7-894b-0723454ee8cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:17.805923 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca5478-fe19-4db7-894b-0723454ee8cc service cinder] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:17.806469 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb0b6dd-ad29-443c-ad1e-d717933f5f7b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:17.807199 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca5478-fe19-4db7-894b-0723454ee8cc service cinder] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:17.807374 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb0b6dd-ad29-443c-ad1e-d717933f5f7b service nova] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:17.807516 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca5478-fe19-4db7-894b-0723454ee8cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:17.807838 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb0b6dd-ad29-443c-ad1e-d717933f5f7b service nova] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:17.808287 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb0b6dd-ad29-443c-ad1e-d717933f5f7b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:17.827551 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb0b6dd-ad29-443c-ad1e-d717933f5f7b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:17.827551 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb0b6dd-ad29-443c-ad1e-d717933f5f7b service nova] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:17.835751 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 920/1829] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:21:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 02 19:21:17.906714 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dca5478-fe19-4db7-894b-0723454ee8cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:17.908770 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dca5478-fe19-4db7-894b-0723454ee8cc service cinder] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:17.923063 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 910/1830] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:21:17 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:21:18.014258 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258b70dc-447a-4dff-8783-8b4b3996a6c3 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:18.014975 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258b70dc-447a-4dff-8783-8b4b3996a6c3 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:18.015705 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258b70dc-447a-4dff-8783-8b4b3996a6c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:18.052503 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-258b70dc-447a-4dff-8783-8b4b3996a6c3 None None] MFA Rules not processed for user `b5337f7eeef24e34a79ef8c6ba4d027e`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:18.054794 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-258b70dc-447a-4dff-8783-8b4b3996a6c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:18.135470 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 921/1831] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:18 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:21:18.334019 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dca5478-fe19-4db7-894b-0723454ee8cc service cinder] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:18.334019 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-7dca5478-fe19-4db7-894b-0723454ee8cc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:18.344584 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dca5478-fe19-4db7-894b-0723454ee8cc service cinder] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:18.348311 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8920a918-a894-41fb-ae46-d5287e347766 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:18.348962 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8920a918-a894-41fb-ae46-d5287e347766 service cinder] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:18.349255 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8920a918-a894-41fb-ae46-d5287e347766 service cinder] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:18.349532 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8920a918-a894-41fb-ae46-d5287e347766 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:18.351453 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-8920a918-a894-41fb-ae46-d5287e347766 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:18.396298 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c7faf-ae75-41fb-bd4c-4bc7473fed9f None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:18.396595 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c7faf-ae75-41fb-bd4c-4bc7473fed9f None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:18.396899 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c7faf-ae75-41fb-bd4c-4bc7473fed9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:18.460501 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8920a918-a894-41fb-ae46-d5287e347766 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:18.464943 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8920a918-a894-41fb-ae46-d5287e347766 service cinder] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:18.484238 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-bc2c7faf-ae75-41fb-bd4c-4bc7473fed9f None None] MFA Rules not processed for user `e6fe13ff783f4171957762a1daff2f52`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:18.487678 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-bc2c7faf-ae75-41fb-bd4c-4bc7473fed9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:18.489592 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 911/1832] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:21:18 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:21:18.596364 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 922/1833] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:18 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:21:18.719992 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:18.814182 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8920a918-a894-41fb-ae46-d5287e347766 service cinder] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:18.815748 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-8920a918-a894-41fb-ae46-d5287e347766 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:18.905765 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8920a918-a894-41fb-ae46-d5287e347766 service cinder] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:18.911289 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3180337a-f1ee-47da-991e-66f4ba394867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:18.916083 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180337a-f1ee-47da-991e-66f4ba394867 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:18.916083 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180337a-f1ee-47da-991e-66f4ba394867 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:18.916083 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180337a-f1ee-47da-991e-66f4ba394867 admin admin] PATH_INFO: `/v3/users/8afd12d701ac4fabb3d3f18a6cc76833` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:18.916981 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3180337a-f1ee-47da-991e-66f4ba394867 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8afd12d701ac4fabb3d3f18a6cc76833)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:18.919155 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3180337a-f1ee-47da-991e-66f4ba394867 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:19.155339 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-3180337a-f1ee-47da-991e-66f4ba394867 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8afd12d701ac4fabb3d3f18a6cc76833'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:19.164711 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-3180337a-f1ee-47da-991e-66f4ba394867 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8afd12d701ac4fabb3d3f18a6cc76833'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:19.165392 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-3180337a-f1ee-47da-991e-66f4ba394867 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8afd12d701ac4fabb3d3f18a6cc76833'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:19.238149 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-3180337a-f1ee-47da-991e-66f4ba394867 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8afd12d701ac4fabb3d3f18a6cc76833'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:19.239176 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-3180337a-f1ee-47da-991e-66f4ba394867 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8afd12d701ac4fabb3d3f18a6cc76833'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:19.259237 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 912/1834] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:21:18 2022] DELETE /identity/v3/users/8afd12d701ac4fabb3d3f18a6cc76833 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:19.263998 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1051c709-45a6-441a-aeae-185838981a93 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:19.264294 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1051c709-45a6-441a-aeae-185838981a93 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:19.265063 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1051c709-45a6-441a-aeae-185838981a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:19.287040 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-1051c709-45a6-441a-aeae-185838981a93 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:19.288785 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-1051c709-45a6-441a-aeae-185838981a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:19.402634 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 923/1835] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:21:19.420508 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3180337a-f1ee-47da-991e-66f4ba394867 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:19.422692 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-3180337a-f1ee-47da-991e-66f4ba394867 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:19.499983 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3180337a-f1ee-47da-991e-66f4ba394867 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:19.505371 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97ef2d9-d4e4-4e6e-bed7-a3f63f602d2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:19.506198 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97ef2d9-d4e4-4e6e-bed7-a3f63f602d2a service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:19.506480 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97ef2d9-d4e4-4e6e-bed7-a3f63f602d2a service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:19.506778 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97ef2d9-d4e4-4e6e-bed7-a3f63f602d2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:19.512940 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc9df3d-9efe-4cb0-a438-83c2fa503431 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:19.513346 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc9df3d-9efe-4cb0-a438-83c2fa503431 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:19.513474 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc9df3d-9efe-4cb0-a438-83c2fa503431 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:19.514926 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 924/1836] 10.222.0.33 () {54 vars in 921 bytes} [Thu Jun 2 19:21:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 02 19:21:19.521345 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97ef2d9-d4e4-4e6e-bed7-a3f63f602d2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:19.522979 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97ef2d9-d4e4-4e6e-bed7-a3f63f602d2a service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:19.532994 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 913/1837] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:21:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:21:19.561391 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a986e2a6-1adf-4691-85ff-2f14f42ce083 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:19.561719 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a986e2a6-1adf-4691-85ff-2f14f42ce083 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:19.561967 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a986e2a6-1adf-4691-85ff-2f14f42ce083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:19.595516 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-a986e2a6-1adf-4691-85ff-2f14f42ce083 None None] MFA Rules not processed for user `73ae837e99764bebbe7a6e06ac9049e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:19.597016 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-a986e2a6-1adf-4691-85ff-2f14f42ce083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:19.674573 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 925/1838] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Jun 2 19:21:19 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:21:19.708735 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97ef2d9-d4e4-4e6e-bed7-a3f63f602d2a service neutron] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:19.710338 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-a97ef2d9-d4e4-4e6e-bed7-a3f63f602d2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:19.726120 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97ef2d9-d4e4-4e6e-bed7-a3f63f602d2a service neutron] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:19.731942 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef47044-b0f8-4055-b43c-2bead4b33f33 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73ae837e99764bebbe7a6e06ac9049e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:19.733346 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef47044-b0f8-4055-b43c-2bead4b33f33 service placement] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:19.733346 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef47044-b0f8-4055-b43c-2bead4b33f33 service placement] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:19.733346 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef47044-b0f8-4055-b43c-2bead4b33f33 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:19.748116 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef47044-b0f8-4055-b43c-2bead4b33f33 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:19.751105 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef47044-b0f8-4055-b43c-2bead4b33f33 service placement] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:19.769736 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 914/1839] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:21:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:21:20.153772 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:20.412984 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a986e2a6-1adf-4691-85ff-2f14f42ce083 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:20.414551 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-a986e2a6-1adf-4691-85ff-2f14f42ce083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:20.491974 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a986e2a6-1adf-4691-85ff-2f14f42ce083 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:20.496619 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dbaf28-24c6-4651-bfae-cb18c8ae477d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:20.497523 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dbaf28-24c6-4651-bfae-cb18c8ae477d service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:20.498278 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dbaf28-24c6-4651-bfae-cb18c8ae477d service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:20.500311 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dbaf28-24c6-4651-bfae-cb18c8ae477d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:20.503018 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-95dbaf28-24c6-4651-bfae-cb18c8ae477d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:20.580673 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dbaf28-24c6-4651-bfae-cb18c8ae477d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:20.582415 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dbaf28-24c6-4651-bfae-cb18c8ae477d service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:20.598609 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 926/1840] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:21:20 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:21:20.821367 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:21.240110 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef47044-b0f8-4055-b43c-2bead4b33f33 service placement] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:21.241850 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-bef47044-b0f8-4055-b43c-2bead4b33f33 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:21.267332 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef47044-b0f8-4055-b43c-2bead4b33f33 service placement] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:21.272154 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344837f6-45bd-441e-a3c0-d24d8878a412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:21.273190 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344837f6-45bd-441e-a3c0-d24d8878a412 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:21.273474 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344837f6-45bd-441e-a3c0-d24d8878a412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:21.273908 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344837f6-45bd-441e-a3c0-d24d8878a412 admin admin] PATH_INFO: `/v3/projects/d85f2cc06c80451bb5ced17157ac0271` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:21.275208 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344837f6-45bd-441e-a3c0-d24d8878a412 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d85f2cc06c80451bb5ced17157ac0271)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:21.277454 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344837f6-45bd-441e-a3c0-d24d8878a412 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:21.343335 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dbaf28-24c6-4651-bfae-cb18c8ae477d service cinder] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:21.353061 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-95dbaf28-24c6-4651-bfae-cb18c8ae477d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:21.373549 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dbaf28-24c6-4651-bfae-cb18c8ae477d service cinder] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:21.379912 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dad2e7-df9e-4d1f-ad53-99c8823ed489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:21.383330 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dad2e7-df9e-4d1f-ad53-99c8823ed489 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:21.384140 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dad2e7-df9e-4d1f-ad53-99c8823ed489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:21.384517 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dad2e7-df9e-4d1f-ad53-99c8823ed489 admin admin] PATH_INFO: `/v3/users/35d9904f560f455dbcc07ef0436d1a69` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:21.387961 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dad2e7-df9e-4d1f-ad53-99c8823ed489 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35d9904f560f455dbcc07ef0436d1a69)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:21.391012 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dad2e7-df9e-4d1f-ad53-99c8823ed489 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:21.436150 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-344837f6-45bd-441e-a3c0-d24d8878a412 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd85f2cc06c80451bb5ced17157ac0271', 'request_id': 'req-344837f6-45bd-441e-a3c0-d24d8878a412'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:21.542128 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-344837f6-45bd-441e-a3c0-d24d8878a412 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd85f2cc06c80451bb5ced17157ac0271', 'request_id': 'req-344837f6-45bd-441e-a3c0-d24d8878a412'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:21.550096 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-344837f6-45bd-441e-a3c0-d24d8878a412 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd85f2cc06c80451bb5ced17157ac0271', 'request_id': 'req-344837f6-45bd-441e-a3c0-d24d8878a412'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:21.563588 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-344837f6-45bd-441e-a3c0-d24d8878a412 admin admin] The token cache is being invalidate because project d85f2cc06c80451bb5ced17157ac0271 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:21:21.564041 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-344837f6-45bd-441e-a3c0-d24d8878a412 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:21.564930 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-344837f6-45bd-441e-a3c0-d24d8878a412 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:21.567815 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 915/1841] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:21:21 2022] DELETE /identity/v3/projects/d85f2cc06c80451bb5ced17157ac0271 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:21.571583 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344837f6-45bd-441e-a3c0-d24d8878a412 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:21.573985 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-344837f6-45bd-441e-a3c0-d24d8878a412 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:21.657226 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344837f6-45bd-441e-a3c0-d24d8878a412 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:21.667569 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2014563c-2909-4a7b-bb6e-e56ca4cf7d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:21.668657 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2014563c-2909-4a7b-bb6e-e56ca4cf7d67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:21.669070 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2014563c-2909-4a7b-bb6e-e56ca4cf7d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:21.669358 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2014563c-2909-4a7b-bb6e-e56ca4cf7d67 admin admin] PATH_INFO: `/v3/users/e43f779027a5494ab65f7852e95bc328` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:21.671176 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2014563c-2909-4a7b-bb6e-e56ca4cf7d67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e43f779027a5494ab65f7852e95bc328)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:21.673440 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2014563c-2909-4a7b-bb6e-e56ca4cf7d67 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:21.676479 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f3dad2e7-df9e-4d1f-ad53-99c8823ed489 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35d9904f560f455dbcc07ef0436d1a69'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:21.690277 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f3dad2e7-df9e-4d1f-ad53-99c8823ed489 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35d9904f560f455dbcc07ef0436d1a69'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:21.759192 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:21.787329 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f3dad2e7-df9e-4d1f-ad53-99c8823ed489 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35d9904f560f455dbcc07ef0436d1a69'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:21.788000 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f3dad2e7-df9e-4d1f-ad53-99c8823ed489 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35d9904f560f455dbcc07ef0436d1a69'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:21.788775 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f3dad2e7-df9e-4d1f-ad53-99c8823ed489 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35d9904f560f455dbcc07ef0436d1a69'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:21.809510 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 927/1842] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:21:21 2022] DELETE /identity/v3/users/35d9904f560f455dbcc07ef0436d1a69 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:21.891009 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-2014563c-2909-4a7b-bb6e-e56ca4cf7d67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e43f779027a5494ab65f7852e95bc328'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:21.901530 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-2014563c-2909-4a7b-bb6e-e56ca4cf7d67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e43f779027a5494ab65f7852e95bc328'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:21.906447 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-2014563c-2909-4a7b-bb6e-e56ca4cf7d67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e43f779027a5494ab65f7852e95bc328'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:21.980523 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dad2e7-df9e-4d1f-ad53-99c8823ed489 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:21.983529 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-f3dad2e7-df9e-4d1f-ad53-99c8823ed489 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:21.999756 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-2014563c-2909-4a7b-bb6e-e56ca4cf7d67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e43f779027a5494ab65f7852e95bc328'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:22.012367 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-2014563c-2909-4a7b-bb6e-e56ca4cf7d67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e43f779027a5494ab65f7852e95bc328'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:22.047737 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 916/1843] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:21:21 2022] DELETE /identity/v3/users/e43f779027a5494ab65f7852e95bc328 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:22.092322 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dad2e7-df9e-4d1f-ad53-99c8823ed489 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:22.120816 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2014563c-2909-4a7b-bb6e-e56ca4cf7d67 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:22.124395 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-2014563c-2909-4a7b-bb6e-e56ca4cf7d67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:22.212838 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:22.257474 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66220f84-77cb-4f7c-b221-d18b9dc34092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:22.266041 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66220f84-77cb-4f7c-b221-d18b9dc34092 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:22.266041 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66220f84-77cb-4f7c-b221-d18b9dc34092 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:22.266041 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66220f84-77cb-4f7c-b221-d18b9dc34092 admin admin] PATH_INFO: `/v3/users/8160092e2ab84839ba807f746fffc570` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:22.266367 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2014563c-2909-4a7b-bb6e-e56ca4cf7d67 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:22.266459 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66220f84-77cb-4f7c-b221-d18b9dc34092 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8160092e2ab84839ba807f746fffc570)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:22.272763 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f9cae2-0829-4e59-b5a8-353ca85afe6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:22.273672 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f9cae2-0829-4e59-b5a8-353ca85afe6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:22.273862 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66220f84-77cb-4f7c-b221-d18b9dc34092 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:22.273971 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f9cae2-0829-4e59-b5a8-353ca85afe6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:22.274221 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f9cae2-0829-4e59-b5a8-353ca85afe6f admin admin] PATH_INFO: `/v3/projects/734454f7f1b949f3bf086a24989f8280` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:22.277617 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f9cae2-0829-4e59-b5a8-353ca85afe6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=734454f7f1b949f3bf086a24989f8280)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:22.294448 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f9cae2-0829-4e59-b5a8-353ca85afe6f admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:22.415879 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:22.469617 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-66220f84-77cb-4f7c-b221-d18b9dc34092 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8160092e2ab84839ba807f746fffc570'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:22.484292 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-00f9cae2-0829-4e59-b5a8-353ca85afe6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '734454f7f1b949f3bf086a24989f8280', 'request_id': 'req-00f9cae2-0829-4e59-b5a8-353ca85afe6f'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:22.530790 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-66220f84-77cb-4f7c-b221-d18b9dc34092 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8160092e2ab84839ba807f746fffc570'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:22.600552 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-00f9cae2-0829-4e59-b5a8-353ca85afe6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '734454f7f1b949f3bf086a24989f8280', 'request_id': 'req-00f9cae2-0829-4e59-b5a8-353ca85afe6f'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:22.608968 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-00f9cae2-0829-4e59-b5a8-353ca85afe6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '734454f7f1b949f3bf086a24989f8280', 'request_id': 'req-00f9cae2-0829-4e59-b5a8-353ca85afe6f'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:22.634482 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-00f9cae2-0829-4e59-b5a8-353ca85afe6f admin admin] The token cache is being invalidate because project 734454f7f1b949f3bf086a24989f8280 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:21:22.635223 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-00f9cae2-0829-4e59-b5a8-353ca85afe6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:22.636429 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-00f9cae2-0829-4e59-b5a8-353ca85afe6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:22.641381 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 917/1844] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:21:22 2022] DELETE /identity/v3/projects/734454f7f1b949f3bf086a24989f8280 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:22.648296 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f9cae2-0829-4e59-b5a8-353ca85afe6f admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:22.652270 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-00f9cae2-0829-4e59-b5a8-353ca85afe6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:22.693035 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-66220f84-77cb-4f7c-b221-d18b9dc34092 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8160092e2ab84839ba807f746fffc570'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:22.693939 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-66220f84-77cb-4f7c-b221-d18b9dc34092 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8160092e2ab84839ba807f746fffc570'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:22.694936 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-66220f84-77cb-4f7c-b221-d18b9dc34092 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8160092e2ab84839ba807f746fffc570'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:22.748967 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 928/1845] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:21:21 2022] DELETE /identity/v3/users/8160092e2ab84839ba807f746fffc570 => generated 0 bytes in 770 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:22.753664 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54057b27-51a0-4a31-9bc6-ebb92cd67f78 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:22.753954 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54057b27-51a0-4a31-9bc6-ebb92cd67f78 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:22.755032 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54057b27-51a0-4a31-9bc6-ebb92cd67f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:22.790296 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f9cae2-0829-4e59-b5a8-353ca85afe6f admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:22.793516 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-54057b27-51a0-4a31-9bc6-ebb92cd67f78 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:22.795356 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-54057b27-51a0-4a31-9bc6-ebb92cd67f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:22.893911 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243ce508-4511-46d8-9894-6d477a2178d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:22.899187 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243ce508-4511-46d8-9894-6d477a2178d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:22.899187 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243ce508-4511-46d8-9894-6d477a2178d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:22.899187 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243ce508-4511-46d8-9894-6d477a2178d4 admin admin] PATH_INFO: `/v3/users/e6fe13ff783f4171957762a1daff2f52` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:22.899187 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243ce508-4511-46d8-9894-6d477a2178d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6fe13ff783f4171957762a1daff2f52)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:22.899509 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243ce508-4511-46d8-9894-6d477a2178d4 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:22.927842 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 929/1846] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:21:22.927842 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54057b27-51a0-4a31-9bc6-ebb92cd67f78 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:22.929217 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-54057b27-51a0-4a31-9bc6-ebb92cd67f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:22.943545 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54057b27-51a0-4a31-9bc6-ebb92cd67f78 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:22.965135 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efb4059-7b1e-4025-8301-49fa798c7943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:22.965135 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb4059-7b1e-4025-8301-49fa798c7943 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:22.965135 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb4059-7b1e-4025-8301-49fa798c7943 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:22.965135 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb4059-7b1e-4025-8301-49fa798c7943 admin admin] PATH_INFO: `/v3/projects/2b8ab5a2d2444c9788905c96e54422ea` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:22.966008 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efb4059-7b1e-4025-8301-49fa798c7943 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b8ab5a2d2444c9788905c96e54422ea)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:22.966008 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efb4059-7b1e-4025-8301-49fa798c7943 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:23.045869 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:23.089183 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-243ce508-4511-46d8-9894-6d477a2178d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6fe13ff783f4171957762a1daff2f52'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:23.096933 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-243ce508-4511-46d8-9894-6d477a2178d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6fe13ff783f4171957762a1daff2f52'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:23.098336 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-243ce508-4511-46d8-9894-6d477a2178d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6fe13ff783f4171957762a1daff2f52'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:23.143208 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-5efb4059-7b1e-4025-8301-49fa798c7943 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b8ab5a2d2444c9788905c96e54422ea', 'request_id': 'req-5efb4059-7b1e-4025-8301-49fa798c7943'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:23.181549 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-243ce508-4511-46d8-9894-6d477a2178d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6fe13ff783f4171957762a1daff2f52'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:23.182409 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-243ce508-4511-46d8-9894-6d477a2178d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6fe13ff783f4171957762a1daff2f52'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:23.205717 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 918/1847] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:21:22 2022] DELETE /identity/v3/users/e6fe13ff783f4171957762a1daff2f52 => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:23.205717 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243ce508-4511-46d8-9894-6d477a2178d4 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:23.205717 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-243ce508-4511-46d8-9894-6d477a2178d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:23.278325 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-5efb4059-7b1e-4025-8301-49fa798c7943 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b8ab5a2d2444c9788905c96e54422ea', 'request_id': 'req-5efb4059-7b1e-4025-8301-49fa798c7943'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:23.285970 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243ce508-4511-46d8-9894-6d477a2178d4 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:23.289811 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-5efb4059-7b1e-4025-8301-49fa798c7943 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b8ab5a2d2444c9788905c96e54422ea', 'request_id': 'req-5efb4059-7b1e-4025-8301-49fa798c7943'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:23.291067 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a99d32-d79b-45c9-bd5d-8485dabc1af9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:23.291992 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a99d32-d79b-45c9-bd5d-8485dabc1af9 service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:23.292277 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a99d32-d79b-45c9-bd5d-8485dabc1af9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:23.292813 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a99d32-d79b-45c9-bd5d-8485dabc1af9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:23.294819 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-05a99d32-d79b-45c9-bd5d-8485dabc1af9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:23.300385 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-5efb4059-7b1e-4025-8301-49fa798c7943 admin admin] The token cache is being invalidate because project 2b8ab5a2d2444c9788905c96e54422ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:21:23.300677 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-5efb4059-7b1e-4025-8301-49fa798c7943 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:23.301535 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-5efb4059-7b1e-4025-8301-49fa798c7943 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:23.303303 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 930/1848] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:21:22 2022] DELETE /identity/v3/projects/2b8ab5a2d2444c9788905c96e54422ea => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:23.308237 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5675ea-61b9-45e0-8cae-9f122f961ed1 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:23.310147 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5675ea-61b9-45e0-8cae-9f122f961ed1 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:23.310147 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5675ea-61b9-45e0-8cae-9f122f961ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:23.339052 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-4f5675ea-61b9-45e0-8cae-9f122f961ed1 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:23.341265 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-4f5675ea-61b9-45e0-8cae-9f122f961ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:23.388809 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a99d32-d79b-45c9-bd5d-8485dabc1af9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:23.390511 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a99d32-d79b-45c9-bd5d-8485dabc1af9 service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:23.404300 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 919/1849] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:21:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:21:23.430785 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 931/1850] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 19:21:23.438804 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf5ee06-551b-4979-961b-8ba35a0dd67a None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:23.439130 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf5ee06-551b-4979-961b-8ba35a0dd67a None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:23.439480 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf5ee06-551b-4979-961b-8ba35a0dd67a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:23.465516 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-caf5ee06-551b-4979-961b-8ba35a0dd67a None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:23.467051 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-caf5ee06-551b-4979-961b-8ba35a0dd67a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:23.477329 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 920/1851] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:21:23.483184 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5675ea-61b9-45e0-8cae-9f122f961ed1 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:23.504757 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5675ea-61b9-45e0-8cae-9f122f961ed1 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:23.530568 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5723d462-f9c6-4516-8c26-4c9434bf39d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:23.531557 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5723d462-f9c6-4516-8c26-4c9434bf39d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:23.531917 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5723d462-f9c6-4516-8c26-4c9434bf39d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:23.532272 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5723d462-f9c6-4516-8c26-4c9434bf39d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:23.533274 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5723d462-f9c6-4516-8c26-4c9434bf39d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:21:23.533815 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5723d462-f9c6-4516-8c26-4c9434bf39d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:23.535604 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5723d462-f9c6-4516-8c26-4c9434bf39d4 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:23.553306 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 932/1852] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 2 19:21:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:21:23.554786 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf5ee06-551b-4979-961b-8ba35a0dd67a None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:23.565413 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf5ee06-551b-4979-961b-8ba35a0dd67a None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:23.572413 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c3e984-a20b-44c4-96e4-b85273ec5515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:23.573353 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c3e984-a20b-44c4-96e4-b85273ec5515 admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:23.573353 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c3e984-a20b-44c4-96e4-b85273ec5515 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:23.574182 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c3e984-a20b-44c4-96e4-b85273ec5515 admin admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:23.574981 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c3e984-a20b-44c4-96e4-b85273ec5515 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:23.579179 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c3e984-a20b-44c4-96e4-b85273ec5515 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:23.735065 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 921/1853] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:21:23 2022] POST /identity/v3/projects => generated 385 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:21:23.742310 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5723d462-f9c6-4516-8c26-4c9434bf39d4 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:23.744065 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:23.760686 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5723d462-f9c6-4516-8c26-4c9434bf39d4 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:23.863700 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ee934e-52c5-4e5f-ae3d-898eb9fc0766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:23.865130 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee934e-52c5-4e5f-ae3d-898eb9fc0766 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:23.866496 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee934e-52c5-4e5f-ae3d-898eb9fc0766 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:23.867008 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee934e-52c5-4e5f-ae3d-898eb9fc0766 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:23.868002 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ee934e-52c5-4e5f-ae3d-898eb9fc0766 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:23.870985 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ee934e-52c5-4e5f-ae3d-898eb9fc0766 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:23.973601 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 933/1854] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:21:23 2022] POST /identity/v3/users => generated 293 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:21:23.978434 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c3e984-a20b-44c4-96e4-b85273ec5515 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:23.990830 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c3e984-a20b-44c4-96e4-b85273ec5515 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:23.996510 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca57b332-a008-487c-b640-b1de84ddba4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:23.997323 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca57b332-a008-487c-b640-b1de84ddba4f admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:23.997711 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca57b332-a008-487c-b640-b1de84ddba4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:23.998794 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca57b332-a008-487c-b640-b1de84ddba4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:23.999121 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca57b332-a008-487c-b640-b1de84ddba4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:24.000944 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca57b332-a008-487c-b640-b1de84ddba4f admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:24.016806 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 922/1855] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:21:23 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:21:24.019641 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ee934e-52c5-4e5f-ae3d-898eb9fc0766 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:24.030877 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ee934e-52c5-4e5f-ae3d-898eb9fc0766 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:24.036357 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f256614-b8d2-4b5d-9168-22a11f38a10d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:24.037156 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f256614-b8d2-4b5d-9168-22a11f38a10d admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:24.037543 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f256614-b8d2-4b5d-9168-22a11f38a10d admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:24.037921 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f256614-b8d2-4b5d-9168-22a11f38a10d admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:24.038972 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f256614-b8d2-4b5d-9168-22a11f38a10d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:24.040690 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f256614-b8d2-4b5d-9168-22a11f38a10d admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:24.052704 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 934/1856] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:21:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 02 19:21:24.057354 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca57b332-a008-487c-b640-b1de84ddba4f admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:24.070676 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f256614-b8d2-4b5d-9168-22a11f38a10d admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:24.071367 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca57b332-a008-487c-b640-b1de84ddba4f admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:24.078959 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-1f256614-b8d2-4b5d-9168-22a11f38a10d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:24.089957 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c259062e-e201-4c0d-9216-08a62bab36d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:24.097872 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c259062e-e201-4c0d-9216-08a62bab36d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:24.097872 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c259062e-e201-4c0d-9216-08a62bab36d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:24.097872 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c259062e-e201-4c0d-9216-08a62bab36d2 admin admin] PATH_INFO: `/v3/projects/ee4cf9c58104436b96ff7212648016a1/users/49c1d4eca65b4dd9b6e11a52f57a33b1/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:24.113719 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c259062e-e201-4c0d-9216-08a62bab36d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee4cf9c58104436b96ff7212648016a1, user_id=49c1d4eca65b4dd9b6e11a52f57a33b1, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:24.119791 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c259062e-e201-4c0d-9216-08a62bab36d2 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:24.129658 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f256614-b8d2-4b5d-9168-22a11f38a10d admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:24.140929 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6422215e-6a78-4cfc-8090-91b091a99395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:24.142019 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422215e-6a78-4cfc-8090-91b091a99395 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:24.142454 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422215e-6a78-4cfc-8090-91b091a99395 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:24.142906 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422215e-6a78-4cfc-8090-91b091a99395 admin admin] PATH_INFO: `/v3/projects/0a3fac8d71564ccc8639a0c9fc15f563` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:24.145640 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6422215e-6a78-4cfc-8090-91b091a99395 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a3fac8d71564ccc8639a0c9fc15f563)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:24.148232 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6422215e-6a78-4cfc-8090-91b091a99395 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:24.217149 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 923/1857] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:21:24 2022] PUT /identity/v3/projects/ee4cf9c58104436b96ff7212648016a1/users/49c1d4eca65b4dd9b6e11a52f57a33b1/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:24.233345 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92943e85-8951-4027-bcd1-fa305f685e5e None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:24.233995 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92943e85-8951-4027-bcd1-fa305f685e5e None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:24.234383 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92943e85-8951-4027-bcd1-fa305f685e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:24.298321 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-92943e85-8951-4027-bcd1-fa305f685e5e None None] MFA Rules not processed for user `49c1d4eca65b4dd9b6e11a52f57a33b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:24.306191 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-92943e85-8951-4027-bcd1-fa305f685e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:24.353272 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-6422215e-6a78-4cfc-8090-91b091a99395 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a3fac8d71564ccc8639a0c9fc15f563', 'request_id': 'req-6422215e-6a78-4cfc-8090-91b091a99395'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:24.438528 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 924/1858] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:21:24.443690 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-6422215e-6a78-4cfc-8090-91b091a99395 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a3fac8d71564ccc8639a0c9fc15f563', 'request_id': 'req-6422215e-6a78-4cfc-8090-91b091a99395'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:24.443851 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92943e85-8951-4027-bcd1-fa305f685e5e None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:24.443851 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-92943e85-8951-4027-bcd1-fa305f685e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:24.449536 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-6422215e-6a78-4cfc-8090-91b091a99395 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a3fac8d71564ccc8639a0c9fc15f563', 'request_id': 'req-6422215e-6a78-4cfc-8090-91b091a99395'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:24.473677 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-6422215e-6a78-4cfc-8090-91b091a99395 admin admin] The token cache is being invalidate because project 0a3fac8d71564ccc8639a0c9fc15f563 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:21:24.473967 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-6422215e-6a78-4cfc-8090-91b091a99395 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:24.480445 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-6422215e-6a78-4cfc-8090-91b091a99395 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:24.485122 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 935/1859] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:21:24 2022] DELETE /identity/v3/projects/0a3fac8d71564ccc8639a0c9fc15f563 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:24.489934 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6422215e-6a78-4cfc-8090-91b091a99395 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:24.495622 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-6422215e-6a78-4cfc-8090-91b091a99395 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:24.543166 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92943e85-8951-4027-bcd1-fa305f685e5e None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:24.598843 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6422215e-6a78-4cfc-8090-91b091a99395 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:24.615548 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5809db-f70e-4ff9-8def-9e2da83a95c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:24.617272 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5809db-f70e-4ff9-8def-9e2da83a95c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:24.617716 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5809db-f70e-4ff9-8def-9e2da83a95c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:24.618188 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5809db-f70e-4ff9-8def-9e2da83a95c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:24.619349 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5809db-f70e-4ff9-8def-9e2da83a95c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:24.623438 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5809db-f70e-4ff9-8def-9e2da83a95c6 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:24.653950 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665dba01-eeee-4934-9766-af1860514961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:24.655004 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665dba01-eeee-4934-9766-af1860514961 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:24.655420 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665dba01-eeee-4934-9766-af1860514961 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:24.655558 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665dba01-eeee-4934-9766-af1860514961 admin admin] PATH_INFO: `/v3/projects/fc9a666a9e7d411fb53b5571866670e7` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:24.656794 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665dba01-eeee-4934-9766-af1860514961 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc9a666a9e7d411fb53b5571866670e7)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:24.658719 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665dba01-eeee-4934-9766-af1860514961 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:24.721611 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 936/1860] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:21:24 2022] POST /identity/v3/projects => generated 387 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:21:24.723779 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5809db-f70e-4ff9-8def-9e2da83a95c6 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:24.728332 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-7e5809db-f70e-4ff9-8def-9e2da83a95c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:24.783358 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-665dba01-eeee-4934-9766-af1860514961 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc9a666a9e7d411fb53b5571866670e7', 'request_id': 'req-665dba01-eeee-4934-9766-af1860514961'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:24.855042 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5809db-f70e-4ff9-8def-9e2da83a95c6 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:24.889840 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-665dba01-eeee-4934-9766-af1860514961 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc9a666a9e7d411fb53b5571866670e7', 'request_id': 'req-665dba01-eeee-4934-9766-af1860514961'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:24.896852 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-665dba01-eeee-4934-9766-af1860514961 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc9a666a9e7d411fb53b5571866670e7', 'request_id': 'req-665dba01-eeee-4934-9766-af1860514961'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:24.929307 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-665dba01-eeee-4934-9766-af1860514961 admin admin] The token cache is being invalidate because project fc9a666a9e7d411fb53b5571866670e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:21:24.930404 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-665dba01-eeee-4934-9766-af1860514961 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:24.932118 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-665dba01-eeee-4934-9766-af1860514961 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:24.938682 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 925/1861] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:21:24 2022] DELETE /identity/v3/projects/fc9a666a9e7d411fb53b5571866670e7 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:24.940220 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665dba01-eeee-4934-9766-af1860514961 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:24.943377 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-665dba01-eeee-4934-9766-af1860514961 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:24.951414 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d51f2a-e9b9-4165-a40a-94357e7fe583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:24.957205 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d51f2a-e9b9-4165-a40a-94357e7fe583 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:24.957903 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d51f2a-e9b9-4165-a40a-94357e7fe583 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:24.958983 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d51f2a-e9b9-4165-a40a-94357e7fe583 admin admin] PATH_INFO: `/v3/users/1d001e561e7f4be0bae2d0b2878b613b` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:24.960426 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d51f2a-e9b9-4165-a40a-94357e7fe583 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d001e561e7f4be0bae2d0b2878b613b)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:24.962839 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d51f2a-e9b9-4165-a40a-94357e7fe583 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:25.038529 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665dba01-eeee-4934-9766-af1860514961 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:25.043914 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c720b5f6-6c91-4735-a1b0-8aa569c99962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:25.044597 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c720b5f6-6c91-4735-a1b0-8aa569c99962 admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:25.044885 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c720b5f6-6c91-4735-a1b0-8aa569c99962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:25.045325 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c720b5f6-6c91-4735-a1b0-8aa569c99962 admin admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:25.046061 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c720b5f6-6c91-4735-a1b0-8aa569c99962 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:25.048586 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c720b5f6-6c91-4735-a1b0-8aa569c99962 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:25.207563 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 926/1862] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:21:24 2022] POST /identity/v3/users => generated 300 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:21:25.209203 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af780023-5402-45f7-85ce-55577e1ed4af None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:25.210040 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af780023-5402-45f7-85ce-55577e1ed4af None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:25.210870 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af780023-5402-45f7-85ce-55577e1ed4af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:25.232866 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c5d51f2a-e9b9-4165-a40a-94357e7fe583 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d001e561e7f4be0bae2d0b2878b613b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:25.240367 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-af780023-5402-45f7-85ce-55577e1ed4af None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:25.241878 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-af780023-5402-45f7-85ce-55577e1ed4af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:25.248159 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c5d51f2a-e9b9-4165-a40a-94357e7fe583 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d001e561e7f4be0bae2d0b2878b613b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:25.267582 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 927/1863] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:21:25.268104 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af780023-5402-45f7-85ce-55577e1ed4af None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:25.285405 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af780023-5402-45f7-85ce-55577e1ed4af None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:25.291416 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ba45f-9a7c-485c-9110-256b8ead1ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:25.292243 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ba45f-9a7c-485c-9110-256b8ead1ed8 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:25.292659 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ba45f-9a7c-485c-9110-256b8ead1ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:25.293039 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ba45f-9a7c-485c-9110-256b8ead1ed8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:25.294004 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71ba45f-9a7c-485c-9110-256b8ead1ed8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:25.297155 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71ba45f-9a7c-485c-9110-256b8ead1ed8 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:25.309227 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 928/1864] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:21:25 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:21:25.310546 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9219f3b-6175-4201-a172-48ece1ebb917 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:25.310858 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9219f3b-6175-4201-a172-48ece1ebb917 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:25.311329 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9219f3b-6175-4201-a172-48ece1ebb917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:25.325493 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c5d51f2a-e9b9-4165-a40a-94357e7fe583 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d001e561e7f4be0bae2d0b2878b613b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:25.326857 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c5d51f2a-e9b9-4165-a40a-94357e7fe583 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d001e561e7f4be0bae2d0b2878b613b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:25.328104 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c5d51f2a-e9b9-4165-a40a-94357e7fe583 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d001e561e7f4be0bae2d0b2878b613b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:25.341449 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-b9219f3b-6175-4201-a172-48ece1ebb917 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:25.342682 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-b9219f3b-6175-4201-a172-48ece1ebb917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:25.352116 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 929/1865] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:21:25.353270 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9219f3b-6175-4201-a172-48ece1ebb917 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:25.354601 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-b9219f3b-6175-4201-a172-48ece1ebb917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:25.367863 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9219f3b-6175-4201-a172-48ece1ebb917 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:25.369193 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 937/1866] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:21:24 2022] DELETE /identity/v3/users/1d001e561e7f4be0bae2d0b2878b613b => generated 0 bytes in 647 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:25.371515 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d51f2a-e9b9-4165-a40a-94357e7fe583 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:25.433925 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57212a4c-d521-4348-b77a-9a4e561d841f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:25.434962 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57212a4c-d521-4348-b77a-9a4e561d841f admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:25.435254 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57212a4c-d521-4348-b77a-9a4e561d841f admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:25.435534 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57212a4c-d521-4348-b77a-9a4e561d841f admin admin] PATH_INFO: `/v3/projects/aad6d64d51924365bfaafa28f7c20844/users/8844397005064672a2cde1774aca99b0/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:25.456991 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57212a4c-d521-4348-b77a-9a4e561d841f admin admin] RBAC: Authorizing `identity:create_grant(project_id=aad6d64d51924365bfaafa28f7c20844, user_id=8844397005064672a2cde1774aca99b0, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:25.459818 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57212a4c-d521-4348-b77a-9a4e561d841f admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:25.522475 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d51f2a-e9b9-4165-a40a-94357e7fe583 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:25.547504 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6a5c9b-8124-420e-888a-7a91e4de3522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:25.548807 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6a5c9b-8124-420e-888a-7a91e4de3522 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:25.549304 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6a5c9b-8124-420e-888a-7a91e4de3522 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:25.550029 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6a5c9b-8124-420e-888a-7a91e4de3522 admin admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:25.553465 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6a5c9b-8124-420e-888a-7a91e4de3522 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:21:25.559170 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6a5c9b-8124-420e-888a-7a91e4de3522 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:25.559170 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6a5c9b-8124-420e-888a-7a91e4de3522 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:25.580027 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 930/1867] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:21:25 2022] PUT /identity/v3/projects/aad6d64d51924365bfaafa28f7c20844/users/8844397005064672a2cde1774aca99b0/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:25.585647 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57212a4c-d521-4348-b77a-9a4e561d841f admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:25.601536 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:25.603147 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 938/1868] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 2 19:21:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:21:25.609902 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6a5c9b-8124-420e-888a-7a91e4de3522 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:25.640625 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6a5c9b-8124-420e-888a-7a91e4de3522 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:25.731679 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57212a4c-d521-4348-b77a-9a4e561d841f admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:25.746536 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dc6b92-ccca-40fe-983f-2538ff4c4d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:25.747426 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc6b92-ccca-40fe-983f-2538ff4c4d43 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:25.747810 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc6b92-ccca-40fe-983f-2538ff4c4d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:25.748308 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc6b92-ccca-40fe-983f-2538ff4c4d43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:25.749407 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dc6b92-ccca-40fe-983f-2538ff4c4d43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:25.753001 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dc6b92-ccca-40fe-983f-2538ff4c4d43 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:25.778183 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 931/1869] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:21:25 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:21:25.784243 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dc6b92-ccca-40fe-983f-2538ff4c4d43 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:25.799316 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:25.805778 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dc6b92-ccca-40fe-983f-2538ff4c4d43 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:25.824420 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b493f0-ab0e-4ea9-a6d1-5c50ed3c2dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:25.828008 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b493f0-ab0e-4ea9-a6d1-5c50ed3c2dc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:25.828008 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b493f0-ab0e-4ea9-a6d1-5c50ed3c2dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:25.828008 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b493f0-ab0e-4ea9-a6d1-5c50ed3c2dc9 admin admin] PATH_INFO: `/v3/projects/aad6d64d51924365bfaafa28f7c20844/users/8844397005064672a2cde1774aca99b0/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:25.834101 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b493f0-ab0e-4ea9-a6d1-5c50ed3c2dc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aad6d64d51924365bfaafa28f7c20844, user_id=8844397005064672a2cde1774aca99b0, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:25.840274 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b493f0-ab0e-4ea9-a6d1-5c50ed3c2dc9 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:25.882823 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 932/1870] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:21:25 2022] PUT /identity/v3/projects/aad6d64d51924365bfaafa28f7c20844/users/8844397005064672a2cde1774aca99b0/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:25.895559 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f6366f-5c79-435b-aae0-3c8c1c40ab1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:25.895559 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f6366f-5c79-435b-aae0-3c8c1c40ab1a admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:25.895559 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f6366f-5c79-435b-aae0-3c8c1c40ab1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:25.895559 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f6366f-5c79-435b-aae0-3c8c1c40ab1a admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:25.896892 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf990817-9e08-4824-bca8-077b0762fe25 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:25.896892 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf990817-9e08-4824-bca8-077b0762fe25 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:25.896892 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf990817-9e08-4824-bca8-077b0762fe25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:25.897451 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f6366f-5c79-435b-aae0-3c8c1c40ab1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:25.913473 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f6366f-5c79-435b-aae0-3c8c1c40ab1a admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:25.995070 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-bf990817-9e08-4824-bca8-077b0762fe25 None None] MFA Rules not processed for user `8844397005064672a2cde1774aca99b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:25.998982 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-bf990817-9e08-4824-bca8-077b0762fe25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:26.071607 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 939/1871] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:21:25 2022] POST /identity/v3/projects => generated 387 bytes in 463 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 02 19:21:26.075754 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f6366f-5c79-435b-aae0-3c8c1c40ab1a admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:26.121607 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 933/1872] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:25 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 19:21:26.136682 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf990817-9e08-4824-bca8-077b0762fe25 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:26.139100 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-bf990817-9e08-4824-bca8-077b0762fe25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:26.177292 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f6366f-5c79-435b-aae0-3c8c1c40ab1a admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:26.182349 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17349a54-5cf0-4b3a-8e8b-b179436b5414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:26.183082 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17349a54-5cf0-4b3a-8e8b-b179436b5414 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:26.183372 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17349a54-5cf0-4b3a-8e8b-b179436b5414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:26.183765 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17349a54-5cf0-4b3a-8e8b-b179436b5414 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:26.184688 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17349a54-5cf0-4b3a-8e8b-b179436b5414 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:26.187412 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17349a54-5cf0-4b3a-8e8b-b179436b5414 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:26.255676 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf990817-9e08-4824-bca8-077b0762fe25 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:26.305456 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 940/1873] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:21:26 2022] POST /identity/v3/users => generated 294 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 02 19:21:26.307362 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17349a54-5cf0-4b3a-8e8b-b179436b5414 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:26.333709 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d14838-8bb5-440a-bb09-8d0c7517df34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:26.334647 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d14838-8bb5-440a-bb09-8d0c7517df34 service cinder] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:26.334966 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d14838-8bb5-440a-bb09-8d0c7517df34 service cinder] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:26.335260 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d14838-8bb5-440a-bb09-8d0c7517df34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:26.338253 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17349a54-5cf0-4b3a-8e8b-b179436b5414 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:26.352582 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7401b9-45c1-4366-917c-24c394856013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:26.353364 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7401b9-45c1-4366-917c-24c394856013 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:26.353833 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7401b9-45c1-4366-917c-24c394856013 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:26.354305 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7401b9-45c1-4366-917c-24c394856013 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:26.355068 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7401b9-45c1-4366-917c-24c394856013 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:26.356998 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7401b9-45c1-4366-917c-24c394856013 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:26.371993 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 941/1874] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:21:26 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:21:26.373399 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7401b9-45c1-4366-917c-24c394856013 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:26.376432 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-9b7401b9-45c1-4366-917c-24c394856013 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:26.391572 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7401b9-45c1-4366-917c-24c394856013 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:26.397495 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a19939c-a0ec-4825-9a40-5757bc4dee55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:26.404692 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a19939c-a0ec-4825-9a40-5757bc4dee55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:26.404692 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a19939c-a0ec-4825-9a40-5757bc4dee55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:26.404692 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a19939c-a0ec-4825-9a40-5757bc4dee55 admin admin] PATH_INFO: `/v3/projects/bec7fef66b7a4145acac655f4d8bbab4` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:26.404692 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a19939c-a0ec-4825-9a40-5757bc4dee55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bec7fef66b7a4145acac655f4d8bbab4)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:26.404692 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a19939c-a0ec-4825-9a40-5757bc4dee55 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:26.445971 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d14838-8bb5-440a-bb09-8d0c7517df34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:26.447481 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d14838-8bb5-440a-bb09-8d0c7517df34 service cinder] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:26.458011 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 934/1875] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:21:26 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 02 19:21:26.458825 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d14838-8bb5-440a-bb09-8d0c7517df34 service cinder] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:26.468898 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d14838-8bb5-440a-bb09-8d0c7517df34 service cinder] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:26.474001 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39720da1-7562-4965-9f4a-bd87f8f6eb12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:26.474669 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39720da1-7562-4965-9f4a-bd87f8f6eb12 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:26.474984 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39720da1-7562-4965-9f4a-bd87f8f6eb12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:26.475265 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39720da1-7562-4965-9f4a-bd87f8f6eb12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:26.479120 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39720da1-7562-4965-9f4a-bd87f8f6eb12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:26.479120 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39720da1-7562-4965-9f4a-bd87f8f6eb12 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:26.488497 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 935/1876] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:21:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:21:26.489333 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39720da1-7562-4965-9f4a-bd87f8f6eb12 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:26.499941 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39720da1-7562-4965-9f4a-bd87f8f6eb12 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:26.505849 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771df3e2-db51-40e6-9aff-e197bc355fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:26.507678 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771df3e2-db51-40e6-9aff-e197bc355fac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:26.507678 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771df3e2-db51-40e6-9aff-e197bc355fac admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:26.507678 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771df3e2-db51-40e6-9aff-e197bc355fac admin admin] PATH_INFO: `/v3/users/49c1d4eca65b4dd9b6e11a52f57a33b1` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:26.508553 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771df3e2-db51-40e6-9aff-e197bc355fac admin admin] RBAC: Authorizing `identity:delete_user(user_id=49c1d4eca65b4dd9b6e11a52f57a33b1)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:26.510532 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771df3e2-db51-40e6-9aff-e197bc355fac admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:26.559217 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-2a19939c-a0ec-4825-9a40-5757bc4dee55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bec7fef66b7a4145acac655f4d8bbab4', 'request_id': 'req-2a19939c-a0ec-4825-9a40-5757bc4dee55'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:26.639372 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-2a19939c-a0ec-4825-9a40-5757bc4dee55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bec7fef66b7a4145acac655f4d8bbab4', 'request_id': 'req-2a19939c-a0ec-4825-9a40-5757bc4dee55'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:26.649639 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-2a19939c-a0ec-4825-9a40-5757bc4dee55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bec7fef66b7a4145acac655f4d8bbab4', 'request_id': 'req-2a19939c-a0ec-4825-9a40-5757bc4dee55'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:26.664245 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-2a19939c-a0ec-4825-9a40-5757bc4dee55 admin admin] The token cache is being invalidate because project bec7fef66b7a4145acac655f4d8bbab4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:21:26.664886 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-2a19939c-a0ec-4825-9a40-5757bc4dee55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:26.666716 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-2a19939c-a0ec-4825-9a40-5757bc4dee55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:26.667924 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 942/1877] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:21:26 2022] DELETE /identity/v3/projects/bec7fef66b7a4145acac655f4d8bbab4 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:26.672069 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a19939c-a0ec-4825-9a40-5757bc4dee55 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:26.674640 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-2a19939c-a0ec-4825-9a40-5757bc4dee55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:26.707207 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-771df3e2-db51-40e6-9aff-e197bc355fac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49c1d4eca65b4dd9b6e11a52f57a33b1'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:26.717982 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-771df3e2-db51-40e6-9aff-e197bc355fac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49c1d4eca65b4dd9b6e11a52f57a33b1'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:26.719666 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-771df3e2-db51-40e6-9aff-e197bc355fac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49c1d4eca65b4dd9b6e11a52f57a33b1'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:26.755047 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a19939c-a0ec-4825-9a40-5757bc4dee55 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:26.760686 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b6e053-cb2e-417b-aef0-7ba97aec5799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:26.761620 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b6e053-cb2e-417b-aef0-7ba97aec5799 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:26.761906 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b6e053-cb2e-417b-aef0-7ba97aec5799 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:26.762187 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b6e053-cb2e-417b-aef0-7ba97aec5799 admin admin] PATH_INFO: `/v3/projects/ca8dbd55907f413fa9d03a8a042ffca7/users/7255dbe6ee46464e8dc0fc43ed848dea/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:26.764713 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-771df3e2-db51-40e6-9aff-e197bc355fac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49c1d4eca65b4dd9b6e11a52f57a33b1'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:26.765625 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-771df3e2-db51-40e6-9aff-e197bc355fac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49c1d4eca65b4dd9b6e11a52f57a33b1'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:26.779452 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b6e053-cb2e-417b-aef0-7ba97aec5799 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca8dbd55907f413fa9d03a8a042ffca7, user_id=7255dbe6ee46464e8dc0fc43ed848dea, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:26.782481 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b6e053-cb2e-417b-aef0-7ba97aec5799 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:26.787358 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 936/1878] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:21:26 2022] DELETE /identity/v3/users/49c1d4eca65b4dd9b6e11a52f57a33b1 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:26.788826 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cb7c9a-3b9a-4420-80e3-52c780e765bd None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:26.789118 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cb7c9a-3b9a-4420-80e3-52c780e765bd None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:26.789744 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cb7c9a-3b9a-4420-80e3-52c780e765bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:26.810851 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-52cb7c9a-3b9a-4420-80e3-52c780e765bd None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:26.812105 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-52cb7c9a-3b9a-4420-80e3-52c780e765bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:26.839062 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 943/1879] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:21:26 2022] PUT /identity/v3/projects/ca8dbd55907f413fa9d03a8a042ffca7/users/7255dbe6ee46464e8dc0fc43ed848dea/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:26.840840 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f40e237-2409-4b96-85a2-59e577e802ef None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:26.841448 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f40e237-2409-4b96-85a2-59e577e802ef None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:26.841715 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f40e237-2409-4b96-85a2-59e577e802ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:26.860415 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-8f40e237-2409-4b96-85a2-59e577e802ef None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:26.861702 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-8f40e237-2409-4b96-85a2-59e577e802ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:26.892130 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 937/1880] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:21:26.893447 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a416173e-b90a-472c-b319-0d0965d736f7 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:26.894365 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a416173e-b90a-472c-b319-0d0965d736f7 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:26.894858 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a416173e-b90a-472c-b319-0d0965d736f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:26.923216 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-a416173e-b90a-472c-b319-0d0965d736f7 None None] MFA Rules not processed for user `7255dbe6ee46464e8dc0fc43ed848dea`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:26.924424 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-a416173e-b90a-472c-b319-0d0965d736f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:26.934078 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 944/1881] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:21:26.935660 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5346591d-bd82-451e-94e3-357e159ca9e5 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:26.935956 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5346591d-bd82-451e-94e3-357e159ca9e5 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:26.936239 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5346591d-bd82-451e-94e3-357e159ca9e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:26.957045 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-5346591d-bd82-451e-94e3-357e159ca9e5 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:26.958270 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-5346591d-bd82-451e-94e3-357e159ca9e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:26.966828 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 945/1882] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:21:26.967650 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5346591d-bd82-451e-94e3-357e159ca9e5 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:26.969185 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-5346591d-bd82-451e-94e3-357e159ca9e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:27.001119 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 938/1883] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:21:27.001846 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a416173e-b90a-472c-b319-0d0965d736f7 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:27.019015 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a416173e-b90a-472c-b319-0d0965d736f7 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:27.024274 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639dd438-65fd-4aa1-814a-01ac5bfb576a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:27.025200 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639dd438-65fd-4aa1-814a-01ac5bfb576a admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:27.025646 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639dd438-65fd-4aa1-814a-01ac5bfb576a admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:27.025890 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639dd438-65fd-4aa1-814a-01ac5bfb576a admin admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:27.027312 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639dd438-65fd-4aa1-814a-01ac5bfb576a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:21:27.027635 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639dd438-65fd-4aa1-814a-01ac5bfb576a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:27.029223 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639dd438-65fd-4aa1-814a-01ac5bfb576a admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:27.043177 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5346591d-bd82-451e-94e3-357e159ca9e5 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:27.043310 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 939/1884] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 2 19:21:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:21:27.043310 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639dd438-65fd-4aa1-814a-01ac5bfb576a admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:27.045128 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-639dd438-65fd-4aa1-814a-01ac5bfb576a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:27.047807 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43200138-461c-4984-be3b-6ce862fd26f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:27.048481 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43200138-461c-4984-be3b-6ce862fd26f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:27.048768 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43200138-461c-4984-be3b-6ce862fd26f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:27.049045 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43200138-461c-4984-be3b-6ce862fd26f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:27.063247 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43200138-461c-4984-be3b-6ce862fd26f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:27.063349 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639dd438-65fd-4aa1-814a-01ac5bfb576a admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:27.063465 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43200138-461c-4984-be3b-6ce862fd26f8 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:27.065301 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d772ffb-72a8-47bf-a5ac-cb1a26cd0a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:27.066241 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d772ffb-72a8-47bf-a5ac-cb1a26cd0a94 admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:27.066527 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d772ffb-72a8-47bf-a5ac-cb1a26cd0a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:27.066836 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d772ffb-72a8-47bf-a5ac-cb1a26cd0a94 admin admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:27.067541 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d772ffb-72a8-47bf-a5ac-cb1a26cd0a94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:27.069455 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d772ffb-72a8-47bf-a5ac-cb1a26cd0a94 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:27.072251 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 946/1885] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:21:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:21:27.072990 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43200138-461c-4984-be3b-6ce862fd26f8 service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:27.082963 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43200138-461c-4984-be3b-6ce862fd26f8 service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:27.087963 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303383b9-e4a6-4d6c-9806-f70f1f0a3885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:27.089926 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303383b9-e4a6-4d6c-9806-f70f1f0a3885 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:27.090214 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303383b9-e4a6-4d6c-9806-f70f1f0a3885 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:27.092083 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303383b9-e4a6-4d6c-9806-f70f1f0a3885 admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:27.092083 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303383b9-e4a6-4d6c-9806-f70f1f0a3885 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:27.093417 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303383b9-e4a6-4d6c-9806-f70f1f0a3885 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:27.170395 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 940/1886] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:21:27 2022] POST /identity/v3/projects => generated 387 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:21:27.171642 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d772ffb-72a8-47bf-a5ac-cb1a26cd0a94 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:27.191937 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 947/1887] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:21:27 2022] POST /identity/v3/projects => generated 375 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:21:27.195925 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303383b9-e4a6-4d6c-9806-f70f1f0a3885 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:27.243215 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d772ffb-72a8-47bf-a5ac-cb1a26cd0a94 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:27.268522 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303383b9-e4a6-4d6c-9806-f70f1f0a3885 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:27.284351 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c960e04-882e-47cb-98d7-c54ae1ec8cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:27.287022 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c960e04-882e-47cb-98d7-c54ae1ec8cc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:27.287670 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c960e04-882e-47cb-98d7-c54ae1ec8cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:27.287971 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c960e04-882e-47cb-98d7-c54ae1ec8cc8 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:27.289202 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c960e04-882e-47cb-98d7-c54ae1ec8cc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:27.291569 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c960e04-882e-47cb-98d7-c54ae1ec8cc8 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:27.322526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452b07c5-c069-426f-b1cf-214781764ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:27.322832 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b07c5-c069-426f-b1cf-214781764ea8 admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:27.323066 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b07c5-c069-426f-b1cf-214781764ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:27.323349 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b07c5-c069-426f-b1cf-214781764ea8 admin admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:27.324059 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452b07c5-c069-426f-b1cf-214781764ea8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:27.326052 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452b07c5-c069-426f-b1cf-214781764ea8 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:27.414795 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 941/1888] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:21:27 2022] POST /identity/v3/users => generated 300 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 02 19:21:27.417905 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452b07c5-c069-426f-b1cf-214781764ea8 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:27.422464 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 948/1889] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:21:27 2022] POST /identity/v3/users => generated 288 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 02 19:21:27.426254 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c960e04-882e-47cb-98d7-c54ae1ec8cc8 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:27.435582 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452b07c5-c069-426f-b1cf-214781764ea8 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:27.442131 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55b9be6-22d9-4f95-a582-89bc7fec5162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:27.442131 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b9be6-22d9-4f95-a582-89bc7fec5162 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:27.442131 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b9be6-22d9-4f95-a582-89bc7fec5162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:27.442363 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b9be6-22d9-4f95-a582-89bc7fec5162 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:27.443268 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c960e04-882e-47cb-98d7-c54ae1ec8cc8 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:27.443395 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55b9be6-22d9-4f95-a582-89bc7fec5162 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:27.444964 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55b9be6-22d9-4f95-a582-89bc7fec5162 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:27.457526 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fae4a23-d1e0-4b76-a26d-cd059c8dedf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:27.458124 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 942/1890] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:21:27 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 02 19:21:27.458244 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae4a23-d1e0-4b76-a26d-cd059c8dedf2 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:27.458495 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae4a23-d1e0-4b76-a26d-cd059c8dedf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:27.458820 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae4a23-d1e0-4b76-a26d-cd059c8dedf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:27.459636 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fae4a23-d1e0-4b76-a26d-cd059c8dedf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:27.463358 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fae4a23-d1e0-4b76-a26d-cd059c8dedf2 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:27.470990 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55b9be6-22d9-4f95-a582-89bc7fec5162 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:27.485193 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55b9be6-22d9-4f95-a582-89bc7fec5162 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:27.491586 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bac8146-a98d-43e1-9685-0a6ac282716b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:27.492086 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 949/1891] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:21:27 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 02 19:21:27.493416 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bac8146-a98d-43e1-9685-0a6ac282716b admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:27.493774 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bac8146-a98d-43e1-9685-0a6ac282716b admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:27.495853 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bac8146-a98d-43e1-9685-0a6ac282716b admin admin] PATH_INFO: `/v3/projects/1597f4b9e0324b6994a864690c931d92/users/6101be2564014529906e9e891d29b228/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:27.495983 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fae4a23-d1e0-4b76-a26d-cd059c8dedf2 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:27.508348 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fae4a23-d1e0-4b76-a26d-cd059c8dedf2 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:27.514190 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05d28b7-c9ec-4e32-a4c2-fbfa82049a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:27.515067 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d28b7-c9ec-4e32-a4c2-fbfa82049a75 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:27.515367 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d28b7-c9ec-4e32-a4c2-fbfa82049a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:27.515838 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d28b7-c9ec-4e32-a4c2-fbfa82049a75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:27.516597 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05d28b7-c9ec-4e32-a4c2-fbfa82049a75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:27.517997 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bac8146-a98d-43e1-9685-0a6ac282716b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1597f4b9e0324b6994a864690c931d92, user_id=6101be2564014529906e9e891d29b228, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:27.518300 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05d28b7-c9ec-4e32-a4c2-fbfa82049a75 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:27.521649 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bac8146-a98d-43e1-9685-0a6ac282716b admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:27.533247 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 950/1892] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:21:27 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:21:27.538344 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05d28b7-c9ec-4e32-a4c2-fbfa82049a75 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:27.549486 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05d28b7-c9ec-4e32-a4c2-fbfa82049a75 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:27.554622 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32011f3-9e93-4631-b16c-3cbe447523b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:27.555982 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32011f3-9e93-4631-b16c-3cbe447523b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:27.556279 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32011f3-9e93-4631-b16c-3cbe447523b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:27.556564 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32011f3-9e93-4631-b16c-3cbe447523b6 admin admin] PATH_INFO: `/v3/projects/912c169ebc1a40ca96b4e31aab39435f/users/6e71a60cc0d8472ebc61d12219f55055/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:27.574328 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32011f3-9e93-4631-b16c-3cbe447523b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=912c169ebc1a40ca96b4e31aab39435f, user_id=6e71a60cc0d8472ebc61d12219f55055, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:27.577027 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32011f3-9e93-4631-b16c-3cbe447523b6 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:27.636916 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 943/1893] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:21:27 2022] PUT /identity/v3/projects/1597f4b9e0324b6994a864690c931d92/users/6101be2564014529906e9e891d29b228/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:27.641136 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bac8146-a98d-43e1-9685-0a6ac282716b admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:27.669319 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 951/1894] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:21:27 2022] PUT /identity/v3/projects/912c169ebc1a40ca96b4e31aab39435f/users/6e71a60cc0d8472ebc61d12219f55055/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 02 19:21:27.670917 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3f307e-d7f3-407a-8a1e-3d4616b27b59 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:27.671329 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3f307e-d7f3-407a-8a1e-3d4616b27b59 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:27.671611 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3f307e-d7f3-407a-8a1e-3d4616b27b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:27.722878 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-ec3f307e-d7f3-407a-8a1e-3d4616b27b59 None None] MFA Rules not processed for user `6e71a60cc0d8472ebc61d12219f55055`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:27.724392 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-ec3f307e-d7f3-407a-8a1e-3d4616b27b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:27.763897 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bac8146-a98d-43e1-9685-0a6ac282716b admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:27.820445 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 952/1895] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:21:27.824844 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3f307e-d7f3-407a-8a1e-3d4616b27b59 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:27.826281 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-ec3f307e-d7f3-407a-8a1e-3d4616b27b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:27.831010 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2487b964-4cf3-4b54-8667-061758461b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:27.832621 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2487b964-4cf3-4b54-8667-061758461b2e admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:27.833090 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2487b964-4cf3-4b54-8667-061758461b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:27.833541 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2487b964-4cf3-4b54-8667-061758461b2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:27.834513 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2487b964-4cf3-4b54-8667-061758461b2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:27.836535 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2487b964-4cf3-4b54-8667-061758461b2e admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:27.842105 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3f307e-d7f3-407a-8a1e-3d4616b27b59 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:27.847607 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d06fce-8e01-4420-a64d-62d71ded63f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:27.848608 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d06fce-8e01-4420-a64d-62d71ded63f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:27.848888 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d06fce-8e01-4420-a64d-62d71ded63f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:27.858914 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d06fce-8e01-4420-a64d-62d71ded63f1 admin admin] PATH_INFO: `/v3/projects/ee4cf9c58104436b96ff7212648016a1` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:27.859058 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 944/1896] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:21:27 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:21:27.859139 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d06fce-8e01-4420-a64d-62d71ded63f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee4cf9c58104436b96ff7212648016a1)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:27.859211 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2487b964-4cf3-4b54-8667-061758461b2e admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:27.859281 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d06fce-8e01-4420-a64d-62d71ded63f1 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:27.866465 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2487b964-4cf3-4b54-8667-061758461b2e admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:27.872163 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eab3d5-efb5-4fca-8f49-55cf08fdfd0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:27.872769 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eab3d5-efb5-4fca-8f49-55cf08fdfd0c admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:27.873054 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eab3d5-efb5-4fca-8f49-55cf08fdfd0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:27.873331 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eab3d5-efb5-4fca-8f49-55cf08fdfd0c admin admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:27.874218 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eab3d5-efb5-4fca-8f49-55cf08fdfd0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:27.877107 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eab3d5-efb5-4fca-8f49-55cf08fdfd0c admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:27.998784 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 945/1897] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:21:27 2022] POST /identity/v3/projects => generated 377 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:21:28.002178 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eab3d5-efb5-4fca-8f49-55cf08fdfd0c admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:28.017469 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-71d06fce-8e01-4420-a64d-62d71ded63f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee4cf9c58104436b96ff7212648016a1', 'request_id': 'req-71d06fce-8e01-4420-a64d-62d71ded63f1'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:28.062835 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eab3d5-efb5-4fca-8f49-55cf08fdfd0c admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:28.068729 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad2dd32-462f-40fe-b0d8-f8ffc606ec1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:28.069636 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad2dd32-462f-40fe-b0d8-f8ffc606ec1d admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:28.069924 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad2dd32-462f-40fe-b0d8-f8ffc606ec1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:28.070210 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad2dd32-462f-40fe-b0d8-f8ffc606ec1d admin admin] PATH_INFO: `/v3/projects/1597f4b9e0324b6994a864690c931d92/users/6101be2564014529906e9e891d29b228/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:28.072636 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad2dd32-462f-40fe-b0d8-f8ffc606ec1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1597f4b9e0324b6994a864690c931d92, user_id=6101be2564014529906e9e891d29b228, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:28.075506 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad2dd32-462f-40fe-b0d8-f8ffc606ec1d admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:28.087737 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 946/1898] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:21:27 2022] PUT /identity/v3/projects/1597f4b9e0324b6994a864690c931d92/users/6101be2564014529906e9e891d29b228/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:28.088567 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad2dd32-462f-40fe-b0d8-f8ffc606ec1d admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:28.120885 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-71d06fce-8e01-4420-a64d-62d71ded63f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee4cf9c58104436b96ff7212648016a1', 'request_id': 'req-71d06fce-8e01-4420-a64d-62d71ded63f1'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:28.129635 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:28.133097 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-71d06fce-8e01-4420-a64d-62d71ded63f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee4cf9c58104436b96ff7212648016a1', 'request_id': 'req-71d06fce-8e01-4420-a64d-62d71ded63f1'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:28.148430 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-71d06fce-8e01-4420-a64d-62d71ded63f1 admin admin] The token cache is being invalidate because project ee4cf9c58104436b96ff7212648016a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:21:28.148872 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-71d06fce-8e01-4420-a64d-62d71ded63f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:28.149771 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-71d06fce-8e01-4420-a64d-62d71ded63f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:28.152623 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 953/1899] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:21:27 2022] DELETE /identity/v3/projects/ee4cf9c58104436b96ff7212648016a1 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:28.157324 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de795d1-a19d-4403-bf8b-d154c9fb634a None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:28.157622 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de795d1-a19d-4403-bf8b-d154c9fb634a None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:28.157910 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de795d1-a19d-4403-bf8b-d154c9fb634a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:28.176469 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad2dd32-462f-40fe-b0d8-f8ffc606ec1d admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:28.182144 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0352196c-5fcc-4aa8-a4d1-2f413e490325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:28.182882 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352196c-5fcc-4aa8-a4d1-2f413e490325 admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:28.183170 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352196c-5fcc-4aa8-a4d1-2f413e490325 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:28.183446 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352196c-5fcc-4aa8-a4d1-2f413e490325 admin admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:28.184146 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0352196c-5fcc-4aa8-a4d1-2f413e490325 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:28.186123 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0352196c-5fcc-4aa8-a4d1-2f413e490325 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:28.188443 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-5de795d1-a19d-4403-bf8b-d154c9fb634a None None] MFA Rules not processed for user `6101be2564014529906e9e891d29b228`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:28.189662 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-5de795d1-a19d-4403-bf8b-d154c9fb634a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:28.270900 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 954/1900] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:28 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:21:28.270900 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de795d1-a19d-4403-bf8b-d154c9fb634a None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:28.270900 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-5de795d1-a19d-4403-bf8b-d154c9fb634a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:28.285835 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de795d1-a19d-4403-bf8b-d154c9fb634a None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:28.290712 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108e3780-f879-4563-ba95-cc1b32c80e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:28.291651 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108e3780-f879-4563-ba95-cc1b32c80e24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:28.291938 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108e3780-f879-4563-ba95-cc1b32c80e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:28.292216 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108e3780-f879-4563-ba95-cc1b32c80e24 admin admin] PATH_INFO: `/v3/users/8844397005064672a2cde1774aca99b0` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:28.293419 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108e3780-f879-4563-ba95-cc1b32c80e24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8844397005064672a2cde1774aca99b0)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:28.295466 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108e3780-f879-4563-ba95-cc1b32c80e24 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:28.333697 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 947/1901] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:21:28 2022] POST /identity/v3/users => generated 295 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:21:28.333697 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0352196c-5fcc-4aa8-a4d1-2f413e490325 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:28.333697 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-0352196c-5fcc-4aa8-a4d1-2f413e490325 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:28.389047 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0352196c-5fcc-4aa8-a4d1-2f413e490325 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:28.392745 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0642b8b4-ca85-43e4-b3f5-9c13683b524c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:28.393408 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0642b8b4-ca85-43e4-b3f5-9c13683b524c service cinder] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:28.393691 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0642b8b4-ca85-43e4-b3f5-9c13683b524c service cinder] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:28.393967 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0642b8b4-ca85-43e4-b3f5-9c13683b524c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:28.403453 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0642b8b4-ca85-43e4-b3f5-9c13683b524c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:28.404898 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0642b8b4-ca85-43e4-b3f5-9c13683b524c service cinder] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:28.415329 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 948/1902] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:21:28 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:21:28.416260 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0642b8b4-ca85-43e4-b3f5-9c13683b524c service cinder] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:28.417708 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-0642b8b4-ca85-43e4-b3f5-9c13683b524c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:28.433526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0642b8b4-ca85-43e4-b3f5-9c13683b524c service cinder] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:28.438765 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb9b59f-5564-4c77-bab9-713fd51c0960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:28.439448 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb9b59f-5564-4c77-bab9-713fd51c0960 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:28.439733 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb9b59f-5564-4c77-bab9-713fd51c0960 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:28.440018 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb9b59f-5564-4c77-bab9-713fd51c0960 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:28.440747 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb9b59f-5564-4c77-bab9-713fd51c0960 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:28.443170 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb9b59f-5564-4c77-bab9-713fd51c0960 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:28.456586 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 949/1903] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:21:28 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:21:28.464722 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb9b59f-5564-4c77-bab9-713fd51c0960 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:28.477365 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb9b59f-5564-4c77-bab9-713fd51c0960 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:28.480663 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42e0a70-72f6-444c-afa8-f58c8c001b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:28.481761 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42e0a70-72f6-444c-afa8-f58c8c001b0e admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:28.482050 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42e0a70-72f6-444c-afa8-f58c8c001b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:28.482332 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42e0a70-72f6-444c-afa8-f58c8c001b0e admin admin] PATH_INFO: `/v3/projects/56234860edf24fcfa3784c7e29e3d3e5/users/37421de1be0446d4a9be4452aed13b9b/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:28.498676 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42e0a70-72f6-444c-afa8-f58c8c001b0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=56234860edf24fcfa3784c7e29e3d3e5, user_id=37421de1be0446d4a9be4452aed13b9b, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:28.501656 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42e0a70-72f6-444c-afa8-f58c8c001b0e admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:28.648663 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:28.650975 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 950/1904] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:21:28 2022] PUT /identity/v3/projects/56234860edf24fcfa3784c7e29e3d3e5/users/37421de1be0446d4a9be4452aed13b9b/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 02 19:21:28.654009 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42e0a70-72f6-444c-afa8-f58c8c001b0e admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:28.656954 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-d42e0a70-72f6-444c-afa8-f58c8c001b0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:28.666415 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-108e3780-f879-4563-ba95-cc1b32c80e24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8844397005064672a2cde1774aca99b0'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:28.674433 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-108e3780-f879-4563-ba95-cc1b32c80e24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8844397005064672a2cde1774aca99b0'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:28.727370 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42e0a70-72f6-444c-afa8-f58c8c001b0e admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:28.732340 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d7973e-b40d-4ead-bd9d-2d75cbcec121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:28.733273 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d7973e-b40d-4ead-bd9d-2d75cbcec121 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:28.733562 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d7973e-b40d-4ead-bd9d-2d75cbcec121 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:28.733842 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d7973e-b40d-4ead-bd9d-2d75cbcec121 admin admin] PATH_INFO: `/v3/users/7255dbe6ee46464e8dc0fc43ed848dea` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:28.735281 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d7973e-b40d-4ead-bd9d-2d75cbcec121 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7255dbe6ee46464e8dc0fc43ed848dea)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:28.737266 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d7973e-b40d-4ead-bd9d-2d75cbcec121 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:28.770038 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-108e3780-f879-4563-ba95-cc1b32c80e24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8844397005064672a2cde1774aca99b0'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:28.770736 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-108e3780-f879-4563-ba95-cc1b32c80e24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8844397005064672a2cde1774aca99b0'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:28.771400 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-108e3780-f879-4563-ba95-cc1b32c80e24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8844397005064672a2cde1774aca99b0'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:28.792067 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 955/1905] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:21:28 2022] DELETE /identity/v3/users/8844397005064672a2cde1774aca99b0 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:28.792931 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108e3780-f879-4563-ba95-cc1b32c80e24 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:28.794396 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-108e3780-f879-4563-ba95-cc1b32c80e24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:28.868172 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108e3780-f879-4563-ba95-cc1b32c80e24 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:28.873528 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813b6ef2-4dd4-439d-ac3b-e36f4b2bb645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:28.874235 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813b6ef2-4dd4-439d-ac3b-e36f4b2bb645 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:28.878279 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813b6ef2-4dd4-439d-ac3b-e36f4b2bb645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:28.878279 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813b6ef2-4dd4-439d-ac3b-e36f4b2bb645 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:28.878279 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813b6ef2-4dd4-439d-ac3b-e36f4b2bb645 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:28.878279 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813b6ef2-4dd4-439d-ac3b-e36f4b2bb645 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:28.887675 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 956/1906] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:21:28 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:21:28.891808 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813b6ef2-4dd4-439d-ac3b-e36f4b2bb645 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:28.902365 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813b6ef2-4dd4-439d-ac3b-e36f4b2bb645 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:28.907238 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db789ef-3028-40fa-952d-f176e8f6aa15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:28.908114 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db789ef-3028-40fa-952d-f176e8f6aa15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:28.908399 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db789ef-3028-40fa-952d-f176e8f6aa15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:28.908678 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db789ef-3028-40fa-952d-f176e8f6aa15 admin admin] PATH_INFO: `/v3/projects/56234860edf24fcfa3784c7e29e3d3e5/users/37421de1be0446d4a9be4452aed13b9b/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:28.911575 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db789ef-3028-40fa-952d-f176e8f6aa15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56234860edf24fcfa3784c7e29e3d3e5, user_id=37421de1be0446d4a9be4452aed13b9b, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:28.914232 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db789ef-3028-40fa-952d-f176e8f6aa15 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:28.924368 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 957/1907] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:21:28 2022] PUT /identity/v3/projects/56234860edf24fcfa3784c7e29e3d3e5/users/37421de1be0446d4a9be4452aed13b9b/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 02 19:21:28.935086 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7b1ba6-643b-42a1-8d75-fb54e46dbbe1 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:28.935381 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7b1ba6-643b-42a1-8d75-fb54e46dbbe1 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:28.935662 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7b1ba6-643b-42a1-8d75-fb54e46dbbe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:28.950742 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-96d7973e-b40d-4ead-bd9d-2d75cbcec121 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7255dbe6ee46464e8dc0fc43ed848dea'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:28.958590 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-96d7973e-b40d-4ead-bd9d-2d75cbcec121 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7255dbe6ee46464e8dc0fc43ed848dea'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:28.959304 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-96d7973e-b40d-4ead-bd9d-2d75cbcec121 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7255dbe6ee46464e8dc0fc43ed848dea'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:28.964366 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-9a7b1ba6-643b-42a1-8d75-fb54e46dbbe1 None None] MFA Rules not processed for user `37421de1be0446d4a9be4452aed13b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:28.965592 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-9a7b1ba6-643b-42a1-8d75-fb54e46dbbe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:29.049890 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-96d7973e-b40d-4ead-bd9d-2d75cbcec121 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7255dbe6ee46464e8dc0fc43ed848dea'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:29.050540 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-96d7973e-b40d-4ead-bd9d-2d75cbcec121 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7255dbe6ee46464e8dc0fc43ed848dea'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:29.057895 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 958/1908] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:28 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 19:21:29.071127 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7b1ba6-643b-42a1-8d75-fb54e46dbbe1 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:29.074352 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-9a7b1ba6-643b-42a1-8d75-fb54e46dbbe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:29.076632 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 951/1909] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:21:28 2022] DELETE /identity/v3/users/7255dbe6ee46464e8dc0fc43ed848dea => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:29.081926 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b463e2ef-dc97-482c-9bb9-6b02c09c424e None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:29.082359 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b463e2ef-dc97-482c-9bb9-6b02c09c424e None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:29.083265 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b463e2ef-dc97-482c-9bb9-6b02c09c424e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:29.125371 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-b463e2ef-dc97-482c-9bb9-6b02c09c424e None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:29.132148 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-b463e2ef-dc97-482c-9bb9-6b02c09c424e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:29.186076 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7b1ba6-643b-42a1-8d75-fb54e46dbbe1 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:29.239221 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 952/1910] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:21:29.251229 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b463e2ef-dc97-482c-9bb9-6b02c09c424e None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:29.251980 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6762465-9f7c-4713-9579-1b609d7a33cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:29.252755 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6762465-9f7c-4713-9579-1b609d7a33cd service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:29.253148 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6762465-9f7c-4713-9579-1b609d7a33cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:29.253468 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-b463e2ef-dc97-482c-9bb9-6b02c09c424e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:29.253580 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6762465-9f7c-4713-9579-1b609d7a33cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:29.257029 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-d6762465-9f7c-4713-9579-1b609d7a33cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:29.341177 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b463e2ef-dc97-482c-9bb9-6b02c09c424e None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:29.345032 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0bb9d4-fccb-472b-87e2-476a2d148b65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:29.346048 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0bb9d4-fccb-472b-87e2-476a2d148b65 service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:29.346340 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0bb9d4-fccb-472b-87e2-476a2d148b65 service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:29.346622 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0bb9d4-fccb-472b-87e2-476a2d148b65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:29.359029 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0bb9d4-fccb-472b-87e2-476a2d148b65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:29.360638 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0bb9d4-fccb-472b-87e2-476a2d148b65 service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:29.374060 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 953/1911] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:21:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 02 19:21:29.376919 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6762465-9f7c-4713-9579-1b609d7a33cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:29.378526 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6762465-9f7c-4713-9579-1b609d7a33cd service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:29.389901 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 959/1912] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:21:29 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 02 19:21:29.603860 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0bb9d4-fccb-472b-87e2-476a2d148b65 service neutron] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:29.606110 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-1c0bb9d4-fccb-472b-87e2-476a2d148b65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:29.633753 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0bb9d4-fccb-472b-87e2-476a2d148b65 service neutron] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:29.640581 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ed077c-2874-47dc-8e2f-36e7bd849b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:29.640581 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed077c-2874-47dc-8e2f-36e7bd849b7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:29.640581 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed077c-2874-47dc-8e2f-36e7bd849b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:29.640941 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed077c-2874-47dc-8e2f-36e7bd849b7d admin admin] PATH_INFO: `/v3/projects/aad6d64d51924365bfaafa28f7c20844` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:29.643416 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ed077c-2874-47dc-8e2f-36e7bd849b7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=aad6d64d51924365bfaafa28f7c20844)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:29.644535 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ed077c-2874-47dc-8e2f-36e7bd849b7d admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:29.783654 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-13ed077c-2874-47dc-8e2f-36e7bd849b7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aad6d64d51924365bfaafa28f7c20844', 'request_id': 'req-13ed077c-2874-47dc-8e2f-36e7bd849b7d'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:29.895988 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-13ed077c-2874-47dc-8e2f-36e7bd849b7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aad6d64d51924365bfaafa28f7c20844', 'request_id': 'req-13ed077c-2874-47dc-8e2f-36e7bd849b7d'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:29.907219 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-13ed077c-2874-47dc-8e2f-36e7bd849b7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aad6d64d51924365bfaafa28f7c20844', 'request_id': 'req-13ed077c-2874-47dc-8e2f-36e7bd849b7d'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:29.922520 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-13ed077c-2874-47dc-8e2f-36e7bd849b7d admin admin] The token cache is being invalidate because project aad6d64d51924365bfaafa28f7c20844 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:21:29.922893 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-13ed077c-2874-47dc-8e2f-36e7bd849b7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:29.928140 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-13ed077c-2874-47dc-8e2f-36e7bd849b7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:29.933377 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 954/1913] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:21:29 2022] DELETE /identity/v3/projects/aad6d64d51924365bfaafa28f7c20844 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:29.935323 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb7e27b-0b24-40f8-bde9-9688161bbd8a None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:29.935678 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb7e27b-0b24-40f8-bde9-9688161bbd8a None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:29.935949 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb7e27b-0b24-40f8-bde9-9688161bbd8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:29.943341 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ed077c-2874-47dc-8e2f-36e7bd849b7d admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:29.947803 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-13ed077c-2874-47dc-8e2f-36e7bd849b7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:29.963683 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-ebb7e27b-0b24-40f8-bde9-9688161bbd8a None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:29.964893 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-ebb7e27b-0b24-40f8-bde9-9688161bbd8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:30.053482 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ed077c-2874-47dc-8e2f-36e7bd849b7d admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:30.058967 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e44d94-789b-4086-bd11-09f97dadc470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:30.059850 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e44d94-789b-4086-bd11-09f97dadc470 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:30.060136 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e44d94-789b-4086-bd11-09f97dadc470 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:30.060535 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e44d94-789b-4086-bd11-09f97dadc470 admin admin] PATH_INFO: `/v3/projects/ca8dbd55907f413fa9d03a8a042ffca7` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:30.061928 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e44d94-789b-4086-bd11-09f97dadc470 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca8dbd55907f413fa9d03a8a042ffca7)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:30.065509 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 960/1914] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:21:30.067559 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e44d94-789b-4086-bd11-09f97dadc470 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:30.080682 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67b7194-298f-4cad-859e-28daff01d3b3 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:30.080970 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67b7194-298f-4cad-859e-28daff01d3b3 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:30.081272 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67b7194-298f-4cad-859e-28daff01d3b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:30.100651 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-e67b7194-298f-4cad-859e-28daff01d3b3 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:30.101857 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-e67b7194-298f-4cad-859e-28daff01d3b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:30.109972 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 961/1915] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 19:21:30.115109 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67b7194-298f-4cad-859e-28daff01d3b3 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:30.125353 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67b7194-298f-4cad-859e-28daff01d3b3 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:30.131889 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815cdae2-516e-4464-960e-99e188e89b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:30.132531 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815cdae2-516e-4464-960e-99e188e89b43 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:30.132814 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815cdae2-516e-4464-960e-99e188e89b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:30.133089 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815cdae2-516e-4464-960e-99e188e89b43 admin admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:30.133847 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815cdae2-516e-4464-960e-99e188e89b43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:21:30.134130 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815cdae2-516e-4464-960e-99e188e89b43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:30.135785 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815cdae2-516e-4464-960e-99e188e89b43 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:30.148997 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 962/1916] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 2 19:21:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:21:30.153670 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815cdae2-516e-4464-960e-99e188e89b43 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:30.164951 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815cdae2-516e-4464-960e-99e188e89b43 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:30.171402 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7a7096-1f9f-42f5-a6b3-0e7ddda4a882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:30.172096 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a7096-1f9f-42f5-a6b3-0e7ddda4a882 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:30.172390 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a7096-1f9f-42f5-a6b3-0e7ddda4a882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:30.172816 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a7096-1f9f-42f5-a6b3-0e7ddda4a882 admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:30.173450 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7a7096-1f9f-42f5-a6b3-0e7ddda4a882 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:30.175446 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7a7096-1f9f-42f5-a6b3-0e7ddda4a882 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:30.189964 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-92e44d94-789b-4086-bd11-09f97dadc470 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca8dbd55907f413fa9d03a8a042ffca7', 'request_id': 'req-92e44d94-789b-4086-bd11-09f97dadc470'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:30.263423 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-92e44d94-789b-4086-bd11-09f97dadc470 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca8dbd55907f413fa9d03a8a042ffca7', 'request_id': 'req-92e44d94-789b-4086-bd11-09f97dadc470'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:30.269838 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 963/1917] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:21:30 2022] POST /identity/v3/projects => generated 407 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:21:30.273804 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-92e44d94-789b-4086-bd11-09f97dadc470 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca8dbd55907f413fa9d03a8a042ffca7', 'request_id': 'req-92e44d94-789b-4086-bd11-09f97dadc470'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:30.274132 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7a7096-1f9f-42f5-a6b3-0e7ddda4a882 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:30.288298 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-92e44d94-789b-4086-bd11-09f97dadc470 admin admin] The token cache is being invalidate because project ca8dbd55907f413fa9d03a8a042ffca7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:21:30.288649 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-92e44d94-789b-4086-bd11-09f97dadc470 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:30.289345 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-92e44d94-789b-4086-bd11-09f97dadc470 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:30.290967 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 955/1918] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:21:29 2022] DELETE /identity/v3/projects/ca8dbd55907f413fa9d03a8a042ffca7 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:30.294758 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e44d94-789b-4086-bd11-09f97dadc470 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:30.296184 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-92e44d94-789b-4086-bd11-09f97dadc470 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:30.357612 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7a7096-1f9f-42f5-a6b3-0e7ddda4a882 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:30.362389 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2579e143-0efb-4797-a2ca-5a21c3e84b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:30.365028 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2579e143-0efb-4797-a2ca-5a21c3e84b74 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:30.365319 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2579e143-0efb-4797-a2ca-5a21c3e84b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:30.365602 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2579e143-0efb-4797-a2ca-5a21c3e84b74 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:30.366649 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2579e143-0efb-4797-a2ca-5a21c3e84b74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:30.368623 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2579e143-0efb-4797-a2ca-5a21c3e84b74 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:30.373027 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e44d94-789b-4086-bd11-09f97dadc470 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:30.378520 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1bba66-b976-437f-9b3f-a40ad68b2d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:30.379497 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1bba66-b976-437f-9b3f-a40ad68b2d84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:30.379784 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1bba66-b976-437f-9b3f-a40ad68b2d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:30.380092 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1bba66-b976-437f-9b3f-a40ad68b2d84 admin admin] PATH_INFO: `/v3/users/6101be2564014529906e9e891d29b228` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:30.381488 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1bba66-b976-437f-9b3f-a40ad68b2d84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6101be2564014529906e9e891d29b228)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:30.383456 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1bba66-b976-437f-9b3f-a40ad68b2d84 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:30.444350 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 964/1919] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:21:30 2022] POST /identity/v3/users => generated 304 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:21:30.447855 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2579e143-0efb-4797-a2ca-5a21c3e84b74 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:30.460905 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-2579e143-0efb-4797-a2ca-5a21c3e84b74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:30.474306 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2579e143-0efb-4797-a2ca-5a21c3e84b74 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:30.479365 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea88a26-2042-413b-824d-e7d09402f599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:30.482454 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea88a26-2042-413b-824d-e7d09402f599 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:30.482454 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea88a26-2042-413b-824d-e7d09402f599 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:30.482454 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea88a26-2042-413b-824d-e7d09402f599 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:30.482454 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea88a26-2042-413b-824d-e7d09402f599 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:30.483406 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea88a26-2042-413b-824d-e7d09402f599 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:30.495257 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 965/1920] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:21:30 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:21:30.499752 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea88a26-2042-413b-824d-e7d09402f599 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:30.511253 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea88a26-2042-413b-824d-e7d09402f599 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:30.516439 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43146de3-503d-48a1-82ed-ff40ae3bfef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:30.517132 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43146de3-503d-48a1-82ed-ff40ae3bfef5 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:30.518055 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43146de3-503d-48a1-82ed-ff40ae3bfef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:30.518055 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43146de3-503d-48a1-82ed-ff40ae3bfef5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:30.518618 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43146de3-503d-48a1-82ed-ff40ae3bfef5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:30.521160 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43146de3-503d-48a1-82ed-ff40ae3bfef5 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:30.538278 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 966/1921] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:21:30 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 02 19:21:30.539239 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43146de3-503d-48a1-82ed-ff40ae3bfef5 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:30.550959 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43146de3-503d-48a1-82ed-ff40ae3bfef5 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:30.555883 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6bdf41-35f8-49f6-877c-ea296cfd040f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:30.556986 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6bdf41-35f8-49f6-877c-ea296cfd040f admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:30.557411 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6bdf41-35f8-49f6-877c-ea296cfd040f admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:30.557713 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6bdf41-35f8-49f6-877c-ea296cfd040f admin admin] PATH_INFO: `/v3/projects/eaf4003b9f254006956f71a1e8ff6d49/users/1a09fdca0fb940fa99ec2459e2a5e0ff/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:30.578846 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6bdf41-35f8-49f6-877c-ea296cfd040f admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaf4003b9f254006956f71a1e8ff6d49, user_id=1a09fdca0fb940fa99ec2459e2a5e0ff, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:30.582963 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6bdf41-35f8-49f6-877c-ea296cfd040f admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:30.660497 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 967/1922] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:21:30 2022] PUT /identity/v3/projects/eaf4003b9f254006956f71a1e8ff6d49/users/1a09fdca0fb940fa99ec2459e2a5e0ff/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:30.677301 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b500b8e0-a396-4f86-a8f1-20b430f003bb None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:30.677599 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b500b8e0-a396-4f86-a8f1-20b430f003bb None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:30.677899 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b500b8e0-a396-4f86-a8f1-20b430f003bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:30.689192 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-2a1bba66-b976-437f-9b3f-a40ad68b2d84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6101be2564014529906e9e891d29b228'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:30.695985 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-2a1bba66-b976-437f-9b3f-a40ad68b2d84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6101be2564014529906e9e891d29b228'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:30.696882 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-2a1bba66-b976-437f-9b3f-a40ad68b2d84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6101be2564014529906e9e891d29b228'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:30.738877 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-b500b8e0-a396-4f86-a8f1-20b430f003bb None None] MFA Rules not processed for user `1a09fdca0fb940fa99ec2459e2a5e0ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:30.740168 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-b500b8e0-a396-4f86-a8f1-20b430f003bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:30.821473 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 968/1923] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:30 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:21:30.825979 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b500b8e0-a396-4f86-a8f1-20b430f003bb None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:30.827454 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-b500b8e0-a396-4f86-a8f1-20b430f003bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:30.850156 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-2a1bba66-b976-437f-9b3f-a40ad68b2d84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6101be2564014529906e9e891d29b228'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:30.860678 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-2a1bba66-b976-437f-9b3f-a40ad68b2d84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6101be2564014529906e9e891d29b228'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:30.871101 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 956/1924] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:21:30 2022] DELETE /identity/v3/users/6101be2564014529906e9e891d29b228 => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 02 19:21:30.902266 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b500b8e0-a396-4f86-a8f1-20b430f003bb None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:30.963765 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b02b882-3d33-4c20-b240-91af91efb1bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:30.964375 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b02b882-3d33-4c20-b240-91af91efb1bb admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:30.964660 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b02b882-3d33-4c20-b240-91af91efb1bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:30.964935 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b02b882-3d33-4c20-b240-91af91efb1bb admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:30.965764 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b02b882-3d33-4c20-b240-91af91efb1bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:30.967882 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b02b882-3d33-4c20-b240-91af91efb1bb admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:31.075668 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 969/1925] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:21:30 2022] POST /identity/v3/projects => generated 409 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 02 19:21:31.079762 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1bba66-b976-437f-9b3f-a40ad68b2d84 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:31.092582 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1bba66-b976-437f-9b3f-a40ad68b2d84 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:31.102724 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:31.217662 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71786a35-0412-41b2-a15e-e6d633790c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:31.218719 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71786a35-0412-41b2-a15e-e6d633790c36 admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:31.219186 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71786a35-0412-41b2-a15e-e6d633790c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:31.219788 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71786a35-0412-41b2-a15e-e6d633790c36 admin admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:31.221322 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71786a35-0412-41b2-a15e-e6d633790c36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:31.225720 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71786a35-0412-41b2-a15e-e6d633790c36 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:31.305273 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 957/1926] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:21:31 2022] POST /identity/v3/users => generated 311 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:21:31.309702 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b02b882-3d33-4c20-b240-91af91efb1bb admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:31.321308 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b02b882-3d33-4c20-b240-91af91efb1bb admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:31.326249 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf8b242-072f-413d-8b39-530a19b27c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:31.327046 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf8b242-072f-413d-8b39-530a19b27c61 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:31.330132 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf8b242-072f-413d-8b39-530a19b27c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:31.330132 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf8b242-072f-413d-8b39-530a19b27c61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:31.330132 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf8b242-072f-413d-8b39-530a19b27c61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:31.330580 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf8b242-072f-413d-8b39-530a19b27c61 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:31.341556 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 970/1927] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:21:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:21:31.345994 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71786a35-0412-41b2-a15e-e6d633790c36 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:31.350346 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:31.360498 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71786a35-0412-41b2-a15e-e6d633790c36 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:31.371251 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e45d87-4b2b-40f3-ab67-f68bd3b774a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:31.372734 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e45d87-4b2b-40f3-ab67-f68bd3b774a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:31.373040 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e45d87-4b2b-40f3-ab67-f68bd3b774a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:31.373340 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e45d87-4b2b-40f3-ab67-f68bd3b774a8 admin admin] PATH_INFO: `/v3/projects/a412e0e750c640909f6908d3f8d8dad6/users/6f994c0f7bb044bdb80e14fbbaa64329/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:31.405255 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e45d87-4b2b-40f3-ab67-f68bd3b774a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a412e0e750c640909f6908d3f8d8dad6, user_id=6f994c0f7bb044bdb80e14fbbaa64329, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:31.408030 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e45d87-4b2b-40f3-ab67-f68bd3b774a8 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:31.477969 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf8b242-072f-413d-8b39-530a19b27c61 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:31.480048 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-dcf8b242-072f-413d-8b39-530a19b27c61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:31.497169 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf8b242-072f-413d-8b39-530a19b27c61 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:31.503534 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc8cf0b-b819-479a-93e2-b6b6bceb126d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:31.505035 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc8cf0b-b819-479a-93e2-b6b6bceb126d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:31.505387 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc8cf0b-b819-479a-93e2-b6b6bceb126d admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:31.505731 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc8cf0b-b819-479a-93e2-b6b6bceb126d admin admin] PATH_INFO: `/v3/projects/1597f4b9e0324b6994a864690c931d92` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:31.507393 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc8cf0b-b819-479a-93e2-b6b6bceb126d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1597f4b9e0324b6994a864690c931d92)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:31.509472 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc8cf0b-b819-479a-93e2-b6b6bceb126d admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:31.524977 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 958/1928] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:21:31 2022] PUT /identity/v3/projects/a412e0e750c640909f6908d3f8d8dad6/users/6f994c0f7bb044bdb80e14fbbaa64329/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:31.527472 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e45d87-4b2b-40f3-ab67-f68bd3b774a8 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:31.635899 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e45d87-4b2b-40f3-ab67-f68bd3b774a8 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:31.638955 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b4ef33-60c4-4993-a8f1-018b410e9e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:31.639633 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b4ef33-60c4-4993-a8f1-018b410e9e38 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:31.639918 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b4ef33-60c4-4993-a8f1-018b410e9e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:31.640196 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b4ef33-60c4-4993-a8f1-018b410e9e38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:31.640924 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b4ef33-60c4-4993-a8f1-018b410e9e38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:31.642631 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b4ef33-60c4-4993-a8f1-018b410e9e38 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:31.652938 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 959/1929] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:21:31 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 02 19:21:31.658089 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b4ef33-60c4-4993-a8f1-018b410e9e38 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:31.673768 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b4ef33-60c4-4993-a8f1-018b410e9e38 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:31.681709 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-acc8cf0b-b819-479a-93e2-b6b6bceb126d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1597f4b9e0324b6994a864690c931d92', 'request_id': 'req-acc8cf0b-b819-479a-93e2-b6b6bceb126d'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:31.737796 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f72cf36-8ce0-45dc-af99-5f432a950267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:31.738620 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72cf36-8ce0-45dc-af99-5f432a950267 admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:31.739511 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72cf36-8ce0-45dc-af99-5f432a950267 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:31.739978 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72cf36-8ce0-45dc-af99-5f432a950267 admin admin] PATH_INFO: `/v3/projects/a412e0e750c640909f6908d3f8d8dad6/users/6f994c0f7bb044bdb80e14fbbaa64329/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:31.742288 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f72cf36-8ce0-45dc-af99-5f432a950267 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a412e0e750c640909f6908d3f8d8dad6, user_id=6f994c0f7bb044bdb80e14fbbaa64329, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:31.745128 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f72cf36-8ce0-45dc-af99-5f432a950267 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:31.755664 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 960/1930] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:21:31 2022] PUT /identity/v3/projects/a412e0e750c640909f6908d3f8d8dad6/users/6f994c0f7bb044bdb80e14fbbaa64329/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:31.765992 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-acc8cf0b-b819-479a-93e2-b6b6bceb126d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1597f4b9e0324b6994a864690c931d92', 'request_id': 'req-acc8cf0b-b819-479a-93e2-b6b6bceb126d'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:31.767575 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee3611a-a21a-4254-9469-b6d5785d8b9c None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:31.767879 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee3611a-a21a-4254-9469-b6d5785d8b9c None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:31.768220 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee3611a-a21a-4254-9469-b6d5785d8b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:31.777289 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-acc8cf0b-b819-479a-93e2-b6b6bceb126d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1597f4b9e0324b6994a864690c931d92', 'request_id': 'req-acc8cf0b-b819-479a-93e2-b6b6bceb126d'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:31.799653 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-0ee3611a-a21a-4254-9469-b6d5785d8b9c None None] MFA Rules not processed for user `6f994c0f7bb044bdb80e14fbbaa64329`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:31.801220 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-0ee3611a-a21a-4254-9469-b6d5785d8b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:31.805471 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-acc8cf0b-b819-479a-93e2-b6b6bceb126d admin admin] The token cache is being invalidate because project 1597f4b9e0324b6994a864690c931d92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:21:31.805837 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-acc8cf0b-b819-479a-93e2-b6b6bceb126d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:31.806587 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-acc8cf0b-b819-479a-93e2-b6b6bceb126d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:31.808283 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 971/1931] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:21:31 2022] DELETE /identity/v3/projects/1597f4b9e0324b6994a864690c931d92 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 02 19:21:31.818056 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a032c5-9295-4187-8714-a37b401e1ff9 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:31.818056 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a032c5-9295-4187-8714-a37b401e1ff9 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:31.818056 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a032c5-9295-4187-8714-a37b401e1ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:31.866975 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-66a032c5-9295-4187-8714-a37b401e1ff9 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:31.870568 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-66a032c5-9295-4187-8714-a37b401e1ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:31.982139 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 961/1932] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:31 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 19:21:32.004497 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee3611a-a21a-4254-9469-b6d5785d8b9c None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:32.007017 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-0ee3611a-a21a-4254-9469-b6d5785d8b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:32.021980 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 972/1933] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:21:32.032453 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139bc1be-1084-462e-bbd9-29fab1b5f384 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:32.033326 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139bc1be-1084-462e-bbd9-29fab1b5f384 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:32.033607 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139bc1be-1084-462e-bbd9-29fab1b5f384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:32.060880 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-139bc1be-1084-462e-bbd9-29fab1b5f384 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:32.062110 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-139bc1be-1084-462e-bbd9-29fab1b5f384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:32.071196 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 973/1934] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:21:32.075390 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139bc1be-1084-462e-bbd9-29fab1b5f384 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:32.086441 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139bc1be-1084-462e-bbd9-29fab1b5f384 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:32.095754 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251c47cc-26b0-4046-8292-6f93a5ff4b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:32.096721 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251c47cc-26b0-4046-8292-6f93a5ff4b26 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:32.097007 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251c47cc-26b0-4046-8292-6f93a5ff4b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:32.097282 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251c47cc-26b0-4046-8292-6f93a5ff4b26 admin admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:32.098074 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251c47cc-26b0-4046-8292-6f93a5ff4b26 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:21:32.099028 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251c47cc-26b0-4046-8292-6f93a5ff4b26 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:32.100915 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251c47cc-26b0-4046-8292-6f93a5ff4b26 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:32.119420 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 974/1935] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 2 19:21:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:21:32.120759 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251c47cc-26b0-4046-8292-6f93a5ff4b26 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:32.124978 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee3611a-a21a-4254-9469-b6d5785d8b9c None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:32.132733 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251c47cc-26b0-4046-8292-6f93a5ff4b26 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:32.135014 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e40d10-ba1f-4c79-b987-c454167d212f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:32.135889 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e40d10-ba1f-4c79-b987-c454167d212f service cinder] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:32.136176 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e40d10-ba1f-4c79-b987-c454167d212f service cinder] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:32.136452 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e40d10-ba1f-4c79-b987-c454167d212f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:32.138613 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbb18a6-9730-45be-84c5-4b75666fa645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:32.139610 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbb18a6-9730-45be-84c5-4b75666fa645 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:32.140408 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbb18a6-9730-45be-84c5-4b75666fa645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:32.140876 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbb18a6-9730-45be-84c5-4b75666fa645 admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:32.141834 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbb18a6-9730-45be-84c5-4b75666fa645 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:32.144756 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbb18a6-9730-45be-84c5-4b75666fa645 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:32.147693 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e40d10-ba1f-4c79-b987-c454167d212f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:32.149823 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e40d10-ba1f-4c79-b987-c454167d212f service cinder] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:32.170929 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 962/1936] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:21:32 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 02 19:21:32.243221 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 975/1937] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:21:32 2022] POST /identity/v3/projects => generated 385 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:21:32.244502 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e40d10-ba1f-4c79-b987-c454167d212f service cinder] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:32.259031 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e40d10-ba1f-4c79-b987-c454167d212f service cinder] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:32.324966 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0334b862-d806-4ab5-88aa-5d80b1a9b8ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:32.325660 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0334b862-d806-4ab5-88aa-5d80b1a9b8ce admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:32.327712 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0334b862-d806-4ab5-88aa-5d80b1a9b8ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:32.327712 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0334b862-d806-4ab5-88aa-5d80b1a9b8ce admin admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:32.327712 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0334b862-d806-4ab5-88aa-5d80b1a9b8ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:32.328914 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0334b862-d806-4ab5-88aa-5d80b1a9b8ce admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:32.429321 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 963/1938] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:21:32 2022] POST /identity/v3/users => generated 293 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 02 19:21:32.433679 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbb18a6-9730-45be-84c5-4b75666fa645 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:32.447931 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbb18a6-9730-45be-84c5-4b75666fa645 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:32.453549 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5402127-77e0-4e95-8ff1-3ace828f735c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:32.454427 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5402127-77e0-4e95-8ff1-3ace828f735c admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:32.454806 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5402127-77e0-4e95-8ff1-3ace828f735c admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:32.455059 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5402127-77e0-4e95-8ff1-3ace828f735c admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:32.460788 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5402127-77e0-4e95-8ff1-3ace828f735c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:32.460788 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5402127-77e0-4e95-8ff1-3ace828f735c admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:32.472711 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 976/1939] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:21:32 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:21:32.474370 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0334b862-d806-4ab5-88aa-5d80b1a9b8ce admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:32.487428 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0334b862-d806-4ab5-88aa-5d80b1a9b8ce admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:32.493642 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61108cea-71c3-42be-9055-b632c1fa610e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:32.494435 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61108cea-71c3-42be-9055-b632c1fa610e admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:32.494838 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61108cea-71c3-42be-9055-b632c1fa610e admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:32.495199 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61108cea-71c3-42be-9055-b632c1fa610e admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:32.496036 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61108cea-71c3-42be-9055-b632c1fa610e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:32.497793 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61108cea-71c3-42be-9055-b632c1fa610e admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:32.515336 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 964/1940] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:21:32 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 02 19:21:32.517249 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5402127-77e0-4e95-8ff1-3ace828f735c admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:32.529909 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5402127-77e0-4e95-8ff1-3ace828f735c admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:32.537571 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c20d5e4-59c9-4ea5-babc-cfe771d1a292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:32.538485 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c20d5e4-59c9-4ea5-babc-cfe771d1a292 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:32.538797 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c20d5e4-59c9-4ea5-babc-cfe771d1a292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:32.539092 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c20d5e4-59c9-4ea5-babc-cfe771d1a292 admin admin] PATH_INFO: `/v3/projects/44c6b29b0b3542c488e7b9428c61ca4c/users/4e541a37b195462187d9ce3eb4073c38/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:32.558090 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c20d5e4-59c9-4ea5-babc-cfe771d1a292 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44c6b29b0b3542c488e7b9428c61ca4c, user_id=4e541a37b195462187d9ce3eb4073c38, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:32.563237 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c20d5e4-59c9-4ea5-babc-cfe771d1a292 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:32.604518 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61108cea-71c3-42be-9055-b632c1fa610e admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:32.652664 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 977/1941] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:21:32 2022] PUT /identity/v3/projects/44c6b29b0b3542c488e7b9428c61ca4c/users/4e541a37b195462187d9ce3eb4073c38/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:32.662802 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61108cea-71c3-42be-9055-b632c1fa610e admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:32.664603 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a725d268-e25b-4a2e-9dce-f03e912882aa None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:32.664966 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a725d268-e25b-4a2e-9dce-f03e912882aa None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:32.665337 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a725d268-e25b-4a2e-9dce-f03e912882aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:32.693436 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-a725d268-e25b-4a2e-9dce-f03e912882aa None None] MFA Rules not processed for user `4e541a37b195462187d9ce3eb4073c38`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:32.694772 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-a725d268-e25b-4a2e-9dce-f03e912882aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:32.715258 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8265b88-b270-4f31-a94b-742acc1cdc4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:32.715908 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8265b88-b270-4f31-a94b-742acc1cdc4d service cinder] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:32.716195 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8265b88-b270-4f31-a94b-742acc1cdc4d service cinder] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:32.716482 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8265b88-b270-4f31-a94b-742acc1cdc4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:32.718392 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-d8265b88-b270-4f31-a94b-742acc1cdc4d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:32.762239 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 978/1942] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:21:32.766709 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a725d268-e25b-4a2e-9dce-f03e912882aa None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:32.781208 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8265b88-b270-4f31-a94b-742acc1cdc4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:32.782669 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8265b88-b270-4f31-a94b-742acc1cdc4d service cinder] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:32.791916 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 965/1943] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:21:32 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 02 19:21:32.830898 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a725d268-e25b-4a2e-9dce-f03e912882aa None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:32.836266 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450afa40-5f1b-4a0f-bd1c-a6cd19ed9d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:32.836956 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450afa40-5f1b-4a0f-bd1c-a6cd19ed9d8b admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:32.837237 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450afa40-5f1b-4a0f-bd1c-a6cd19ed9d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:32.837516 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450afa40-5f1b-4a0f-bd1c-a6cd19ed9d8b admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:32.838227 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450afa40-5f1b-4a0f-bd1c-a6cd19ed9d8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:32.840197 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450afa40-5f1b-4a0f-bd1c-a6cd19ed9d8b admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:32.965692 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 979/1944] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:21:32 2022] POST /identity/v3/projects => generated 383 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 02 19:21:32.970150 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8265b88-b270-4f31-a94b-742acc1cdc4d service cinder] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:32.981026 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8265b88-b270-4f31-a94b-742acc1cdc4d service cinder] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:33.042550 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454b992b-7dac-4440-9a33-2944ad0a9da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:33.043311 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454b992b-7dac-4440-9a33-2944ad0a9da8 admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:33.043596 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454b992b-7dac-4440-9a33-2944ad0a9da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:33.043872 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454b992b-7dac-4440-9a33-2944ad0a9da8 admin admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:33.044581 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454b992b-7dac-4440-9a33-2944ad0a9da8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:33.046501 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454b992b-7dac-4440-9a33-2944ad0a9da8 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:33.170146 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 966/1945] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:21:32 2022] POST /identity/v3/users => generated 292 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:21:33.174210 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450afa40-5f1b-4a0f-bd1c-a6cd19ed9d8b admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:33.184811 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450afa40-5f1b-4a0f-bd1c-a6cd19ed9d8b admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:33.189250 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124fe852-3f55-41d3-aab1-17831dc81070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:33.189905 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124fe852-3f55-41d3-aab1-17831dc81070 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:33.190187 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124fe852-3f55-41d3-aab1-17831dc81070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:33.190462 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124fe852-3f55-41d3-aab1-17831dc81070 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:33.191395 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124fe852-3f55-41d3-aab1-17831dc81070 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:33.192856 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124fe852-3f55-41d3-aab1-17831dc81070 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:33.204324 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 980/1946] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:21:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:21:33.208057 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454b992b-7dac-4440-9a33-2944ad0a9da8 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:33.218417 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454b992b-7dac-4440-9a33-2944ad0a9da8 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:33.223665 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12139d2b-b709-4672-aebc-4d7dbb8573c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:33.224329 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12139d2b-b709-4672-aebc-4d7dbb8573c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:33.224617 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12139d2b-b709-4672-aebc-4d7dbb8573c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:33.224900 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12139d2b-b709-4672-aebc-4d7dbb8573c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:33.225688 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12139d2b-b709-4672-aebc-4d7dbb8573c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:33.227300 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12139d2b-b709-4672-aebc-4d7dbb8573c4 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:33.237142 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 967/1947] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:21:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:21:33.242391 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124fe852-3f55-41d3-aab1-17831dc81070 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:33.252267 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124fe852-3f55-41d3-aab1-17831dc81070 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:33.256773 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aab6d7-ca64-4f7d-a529-ef2f79625367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:33.257648 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aab6d7-ca64-4f7d-a529-ef2f79625367 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:33.257926 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aab6d7-ca64-4f7d-a529-ef2f79625367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:33.258198 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aab6d7-ca64-4f7d-a529-ef2f79625367 admin admin] PATH_INFO: `/v3/projects/907e8b96738746b9b10db27a679c45d1/users/7b56ba129e384934a10a73af29871413/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:33.272812 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aab6d7-ca64-4f7d-a529-ef2f79625367 admin admin] RBAC: Authorizing `identity:create_grant(project_id=907e8b96738746b9b10db27a679c45d1, user_id=7b56ba129e384934a10a73af29871413, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:33.276467 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aab6d7-ca64-4f7d-a529-ef2f79625367 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:33.376554 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 981/1948] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:21:33 2022] PUT /identity/v3/projects/907e8b96738746b9b10db27a679c45d1/users/7b56ba129e384934a10a73af29871413/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:33.386892 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d2fea5-7bd5-4a62-9101-41fad9257eeb None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:33.387416 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d2fea5-7bd5-4a62-9101-41fad9257eeb None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:33.388016 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d2fea5-7bd5-4a62-9101-41fad9257eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:33.420626 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-e5d2fea5-7bd5-4a62-9101-41fad9257eeb None None] MFA Rules not processed for user `7b56ba129e384934a10a73af29871413`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:33.431268 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-e5d2fea5-7bd5-4a62-9101-41fad9257eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:33.504518 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 968/1949] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:21:33.508818 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aab6d7-ca64-4f7d-a529-ef2f79625367 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:33.571929 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aab6d7-ca64-4f7d-a529-ef2f79625367 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:33.576876 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116ddf10-a13c-497d-9df3-e8988be5bc0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:33.577544 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116ddf10-a13c-497d-9df3-e8988be5bc0a admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:33.577825 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116ddf10-a13c-497d-9df3-e8988be5bc0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:33.578100 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116ddf10-a13c-497d-9df3-e8988be5bc0a admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:33.578825 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116ddf10-a13c-497d-9df3-e8988be5bc0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:33.580731 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116ddf10-a13c-497d-9df3-e8988be5bc0a admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:33.708251 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 982/1950] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:21:33 2022] POST /identity/v3/projects => generated 383 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:21:33.709390 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d2fea5-7bd5-4a62-9101-41fad9257eeb None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:33.719928 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d2fea5-7bd5-4a62-9101-41fad9257eeb None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:33.782595 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af954e6d-7e5c-4260-8a71-a1f2d91a1443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:33.783174 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af954e6d-7e5c-4260-8a71-a1f2d91a1443 admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:33.783581 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af954e6d-7e5c-4260-8a71-a1f2d91a1443 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:33.783749 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af954e6d-7e5c-4260-8a71-a1f2d91a1443 admin admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:33.784622 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af954e6d-7e5c-4260-8a71-a1f2d91a1443 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:33.786738 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af954e6d-7e5c-4260-8a71-a1f2d91a1443 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:33.868167 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 969/1951] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:21:33 2022] POST /identity/v3/users => generated 298 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:21:33.872261 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116ddf10-a13c-497d-9df3-e8988be5bc0a admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:33.883692 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116ddf10-a13c-497d-9df3-e8988be5bc0a admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:33.888597 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cffab8-c828-4cec-bbc5-f4fa16d506e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:33.889275 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cffab8-c828-4cec-bbc5-f4fa16d506e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:33.889690 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cffab8-c828-4cec-bbc5-f4fa16d506e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:33.889975 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cffab8-c828-4cec-bbc5-f4fa16d506e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:33.890715 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cffab8-c828-4cec-bbc5-f4fa16d506e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:33.892373 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cffab8-c828-4cec-bbc5-f4fa16d506e9 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:33.903652 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 983/1952] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:21:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:21:33.907939 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af954e6d-7e5c-4260-8a71-a1f2d91a1443 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:33.918103 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af954e6d-7e5c-4260-8a71-a1f2d91a1443 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:33.922770 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa7f176-07bf-4941-a686-42e310a8bd01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:33.923672 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa7f176-07bf-4941-a686-42e310a8bd01 admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:33.923958 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa7f176-07bf-4941-a686-42e310a8bd01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:33.924235 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa7f176-07bf-4941-a686-42e310a8bd01 admin admin] PATH_INFO: `/v3/projects/6c6cc5f20c4e4df4970550bea9f27282/users/591ec8887c0c4bc1aefcbe6ae4809b13/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:33.939105 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa7f176-07bf-4941-a686-42e310a8bd01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c6cc5f20c4e4df4970550bea9f27282, user_id=591ec8887c0c4bc1aefcbe6ae4809b13, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:33.941765 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa7f176-07bf-4941-a686-42e310a8bd01 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:34.039310 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 970/1953] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:21:33 2022] PUT /identity/v3/projects/6c6cc5f20c4e4df4970550bea9f27282/users/591ec8887c0c4bc1aefcbe6ae4809b13/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:34.043287 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cffab8-c828-4cec-bbc5-f4fa16d506e9 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:34.053716 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cffab8-c828-4cec-bbc5-f4fa16d506e9 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:34.116965 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5822a3e-ce81-4888-9ef2-3184d8baaa35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:34.116965 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5822a3e-ce81-4888-9ef2-3184d8baaa35 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:34.116965 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5822a3e-ce81-4888-9ef2-3184d8baaa35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:34.116965 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5822a3e-ce81-4888-9ef2-3184d8baaa35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:34.117536 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5822a3e-ce81-4888-9ef2-3184d8baaa35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:34.117536 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5822a3e-ce81-4888-9ef2-3184d8baaa35 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:34.130129 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 984/1954] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:21:34 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:21:34.134436 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa7f176-07bf-4941-a686-42e310a8bd01 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:34.144878 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa7f176-07bf-4941-a686-42e310a8bd01 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:34.149861 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0727e8e9-d993-4430-9db7-c1a962ce0b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:34.150775 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0727e8e9-d993-4430-9db7-c1a962ce0b14 admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:34.151685 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0727e8e9-d993-4430-9db7-c1a962ce0b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:34.151968 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0727e8e9-d993-4430-9db7-c1a962ce0b14 admin admin] PATH_INFO: `/v3/projects/6c6cc5f20c4e4df4970550bea9f27282/users/591ec8887c0c4bc1aefcbe6ae4809b13/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:34.153964 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0727e8e9-d993-4430-9db7-c1a962ce0b14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c6cc5f20c4e4df4970550bea9f27282, user_id=591ec8887c0c4bc1aefcbe6ae4809b13, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:34.156609 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0727e8e9-d993-4430-9db7-c1a962ce0b14 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:34.168183 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 971/1955] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:21:34 2022] PUT /identity/v3/projects/6c6cc5f20c4e4df4970550bea9f27282/users/591ec8887c0c4bc1aefcbe6ae4809b13/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:34.177767 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:34.179149 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ace7a7-f283-4670-95bd-3708453f1225 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:34.179587 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ace7a7-f283-4670-95bd-3708453f1225 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:34.179711 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ace7a7-f283-4670-95bd-3708453f1225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:34.212940 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-32ace7a7-f283-4670-95bd-3708453f1225 None None] MFA Rules not processed for user `591ec8887c0c4bc1aefcbe6ae4809b13`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:34.214289 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-32ace7a7-f283-4670-95bd-3708453f1225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:34.250349 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:34.284218 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 985/1956] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:34 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 19:21:34.295766 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0727e8e9-d993-4430-9db7-c1a962ce0b14 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:34.297481 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-0727e8e9-d993-4430-9db7-c1a962ce0b14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:34.354943 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0727e8e9-d993-4430-9db7-c1a962ce0b14 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:34.358864 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e78607-4a57-4cdf-b6cb-01946d2dca4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:34.359515 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e78607-4a57-4cdf-b6cb-01946d2dca4d service cinder] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:34.359843 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e78607-4a57-4cdf-b6cb-01946d2dca4d service cinder] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:34.360126 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e78607-4a57-4cdf-b6cb-01946d2dca4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:34.418002 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e78607-4a57-4cdf-b6cb-01946d2dca4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:34.419408 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e78607-4a57-4cdf-b6cb-01946d2dca4d service cinder] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:34.428804 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 972/1957] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:21:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:21:35.776293 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:35.799813 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:36.396669 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ace7a7-f283-4670-95bd-3708453f1225 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:36.407050 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ace7a7-f283-4670-95bd-3708453f1225 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:36.411047 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efea66a0-7d37-4838-9a05-d6fce0c95d03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:36.413389 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea66a0-7d37-4838-9a05-d6fce0c95d03 service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:36.413389 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea66a0-7d37-4838-9a05-d6fce0c95d03 service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:36.413389 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea66a0-7d37-4838-9a05-d6fce0c95d03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:36.480736 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efea66a0-7d37-4838-9a05-d6fce0c95d03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:36.482193 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efea66a0-7d37-4838-9a05-d6fce0c95d03 service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:36.493297 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 986/1958] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:21:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 02 19:21:36.559736 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e78607-4a57-4cdf-b6cb-01946d2dca4d service cinder] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:36.569594 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e78607-4a57-4cdf-b6cb-01946d2dca4d service cinder] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:36.574327 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1686454-b283-4b0a-ac9c-38a67a32a72b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:36.575318 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1686454-b283-4b0a-ac9c-38a67a32a72b service cinder] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:36.575793 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1686454-b283-4b0a-ac9c-38a67a32a72b service cinder] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:36.576221 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1686454-b283-4b0a-ac9c-38a67a32a72b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:36.578431 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-a1686454-b283-4b0a-ac9c-38a67a32a72b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:36.714237 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1686454-b283-4b0a-ac9c-38a67a32a72b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:36.715725 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1686454-b283-4b0a-ac9c-38a67a32a72b service cinder] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:36.726107 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 973/1959] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:21:36 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:21:36.868800 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:37.393275 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:38.205094 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:38.340130 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efea66a0-7d37-4838-9a05-d6fce0c95d03 service cinder] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:38.351032 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efea66a0-7d37-4838-9a05-d6fce0c95d03 service cinder] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:38.358139 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8604a6-55d4-4373-84e6-156c5d650350 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:38.358831 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8604a6-55d4-4373-84e6-156c5d650350 service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:38.359120 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8604a6-55d4-4373-84e6-156c5d650350 service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:38.359399 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8604a6-55d4-4373-84e6-156c5d650350 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:38.369521 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8604a6-55d4-4373-84e6-156c5d650350 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:38.382075 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8604a6-55d4-4373-84e6-156c5d650350 service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:38.389633 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 987/1960] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:21:38 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:21:38.713632 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:38.907585 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:39.221625 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:39.393000 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:39.599353 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:39.633001 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:39.827519 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:39.878899 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:39.999421 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:40.056280 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:40.176153 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:40.204796 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:40.307685 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:40.324931 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:40.377507 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:40.524814 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:40.667797 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:40.724821 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:40.781984 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:48.461457 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e60dd15-baf9-4c76-a106-c434226cc2fe None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:48.464703 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e60dd15-baf9-4c76-a106-c434226cc2fe None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:48.464703 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e60dd15-baf9-4c76-a106-c434226cc2fe None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:48.464703 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 974/1961] 10.222.0.33 () {54 vars in 921 bytes} [Thu Jun 2 19:21:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:21:48.502011 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fbe2ec-3667-4e35-b3d7-c336055c8b60 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:48.502588 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fbe2ec-3667-4e35-b3d7-c336055c8b60 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:48.502588 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fbe2ec-3667-4e35-b3d7-c336055c8b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:48.528181 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-56fbe2ec-3667-4e35-b3d7-c336055c8b60 None None] MFA Rules not processed for user `73ae837e99764bebbe7a6e06ac9049e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:48.529568 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-56fbe2ec-3667-4e35-b3d7-c336055c8b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:48.615697 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 988/1962] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Jun 2 19:21:48 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:21:48.630028 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e60dd15-baf9-4c76-a106-c434226cc2fe None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:48.631910 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-4e60dd15-baf9-4c76-a106-c434226cc2fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:48.657017 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e60dd15-baf9-4c76-a106-c434226cc2fe None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:48.675947 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f21e607-2c2e-4f9c-980b-8f2bb4edeae4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73ae837e99764bebbe7a6e06ac9049e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:48.676307 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f21e607-2c2e-4f9c-980b-8f2bb4edeae4 service placement] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:48.676683 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f21e607-2c2e-4f9c-980b-8f2bb4edeae4 service placement] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:48.677206 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f21e607-2c2e-4f9c-980b-8f2bb4edeae4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:48.687389 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fbe2ec-3667-4e35-b3d7-c336055c8b60 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:48.688827 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-56fbe2ec-3667-4e35-b3d7-c336055c8b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:48.691375 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f21e607-2c2e-4f9c-980b-8f2bb4edeae4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:48.694389 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f21e607-2c2e-4f9c-980b-8f2bb4edeae4 service placement] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:48.706984 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 975/1963] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:21:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:21:48.718772 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:48.785760 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fbe2ec-3667-4e35-b3d7-c336055c8b60 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:48.802991 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18843c8-e5c5-40ec-a958-281ba792f24a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:48.804042 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18843c8-e5c5-40ec-a958-281ba792f24a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:48.804328 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18843c8-e5c5-40ec-a958-281ba792f24a admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:48.804615 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18843c8-e5c5-40ec-a958-281ba792f24a admin admin] PATH_INFO: `/v3/users/72558099db31474292377bab47035911` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:48.807333 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18843c8-e5c5-40ec-a958-281ba792f24a admin admin] RBAC: Authorizing `identity:delete_user(user_id=72558099db31474292377bab47035911)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:48.816988 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18843c8-e5c5-40ec-a958-281ba792f24a admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:48.818892 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f21e607-2c2e-4f9c-980b-8f2bb4edeae4 service placement] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:48.821031 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-8f21e607-2c2e-4f9c-980b-8f2bb4edeae4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:48.850878 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f21e607-2c2e-4f9c-980b-8f2bb4edeae4 service placement] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:48.859054 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f585799-efd5-425f-877f-447026dcbc89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:48.859982 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f585799-efd5-425f-877f-447026dcbc89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:48.860274 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f585799-efd5-425f-877f-447026dcbc89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:48.860620 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f585799-efd5-425f-877f-447026dcbc89 admin admin] PATH_INFO: `/v3/users/1a09fdca0fb940fa99ec2459e2a5e0ff` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:48.862689 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f585799-efd5-425f-877f-447026dcbc89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a09fdca0fb940fa99ec2459e2a5e0ff)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:48.864713 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f585799-efd5-425f-877f-447026dcbc89 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:49.001556 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:49.084241 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-7f585799-efd5-425f-877f-447026dcbc89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a09fdca0fb940fa99ec2459e2a5e0ff'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:49.085313 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-b18843c8-e5c5-40ec-a958-281ba792f24a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72558099db31474292377bab47035911'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:49.093908 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-b18843c8-e5c5-40ec-a958-281ba792f24a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72558099db31474292377bab47035911'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:49.094236 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-7f585799-efd5-425f-877f-447026dcbc89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a09fdca0fb940fa99ec2459e2a5e0ff'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:49.095164 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-7f585799-efd5-425f-877f-447026dcbc89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a09fdca0fb940fa99ec2459e2a5e0ff'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:49.170494 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-b18843c8-e5c5-40ec-a958-281ba792f24a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72558099db31474292377bab47035911'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:49.171094 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-b18843c8-e5c5-40ec-a958-281ba792f24a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72558099db31474292377bab47035911'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:49.171737 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-b18843c8-e5c5-40ec-a958-281ba792f24a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72558099db31474292377bab47035911'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:49.192935 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 989/1964] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:21:48 2022] DELETE /identity/v3/users/72558099db31474292377bab47035911 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:49.250599 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-7f585799-efd5-425f-877f-447026dcbc89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a09fdca0fb940fa99ec2459e2a5e0ff'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:49.251313 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-7f585799-efd5-425f-877f-447026dcbc89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a09fdca0fb940fa99ec2459e2a5e0ff'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:49.271993 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 976/1965] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:21:48 2022] DELETE /identity/v3/users/1a09fdca0fb940fa99ec2459e2a5e0ff => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 02 19:21:49.278383 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad2d119-5cb6-4539-abd9-6c0599896578 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:49.278684 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad2d119-5cb6-4539-abd9-6c0599896578 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:49.278999 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad2d119-5cb6-4539-abd9-6c0599896578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:49.299722 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-5ad2d119-5cb6-4539-abd9-6c0599896578 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:49.301554 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-5ad2d119-5cb6-4539-abd9-6c0599896578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:49.379942 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 990/1966] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:21:49.392166 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f585799-efd5-425f-877f-447026dcbc89 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:49.393908 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-7f585799-efd5-425f-877f-447026dcbc89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:49.454679 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f585799-efd5-425f-877f-447026dcbc89 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:49.458300 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a48321-94ab-43c2-8ba1-52d0f7e7dc0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:49.459015 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a48321-94ab-43c2-8ba1-52d0f7e7dc0e service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:49.459300 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a48321-94ab-43c2-8ba1-52d0f7e7dc0e service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:49.459580 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a48321-94ab-43c2-8ba1-52d0f7e7dc0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:49.471898 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a48321-94ab-43c2-8ba1-52d0f7e7dc0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:49.473182 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a48321-94ab-43c2-8ba1-52d0f7e7dc0e service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:49.475711 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad2d119-5cb6-4539-abd9-6c0599896578 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:49.477632 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-5ad2d119-5cb6-4539-abd9-6c0599896578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:49.492167 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad2d119-5cb6-4539-abd9-6c0599896578 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:49.495667 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 977/1967] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:21:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:21:49.498339 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b010306-8b0c-4a49-8ffa-c99fdfb8adee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:49.499281 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b010306-8b0c-4a49-8ffa-c99fdfb8adee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:49.504933 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b010306-8b0c-4a49-8ffa-c99fdfb8adee admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:49.504933 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b010306-8b0c-4a49-8ffa-c99fdfb8adee admin admin] PATH_INFO: `/v3/projects/2cb2e1eb3a5e4c0bab176a8e0be12dc9` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:49.504933 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b010306-8b0c-4a49-8ffa-c99fdfb8adee admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cb2e1eb3a5e4c0bab176a8e0be12dc9)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:49.504933 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b010306-8b0c-4a49-8ffa-c99fdfb8adee admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:49.633850 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:49.646207 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-8b010306-8b0c-4a49-8ffa-c99fdfb8adee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cb2e1eb3a5e4c0bab176a8e0be12dc9', 'request_id': 'req-8b010306-8b0c-4a49-8ffa-c99fdfb8adee'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:49.759336 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-8b010306-8b0c-4a49-8ffa-c99fdfb8adee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cb2e1eb3a5e4c0bab176a8e0be12dc9', 'request_id': 'req-8b010306-8b0c-4a49-8ffa-c99fdfb8adee'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:49.773025 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-8b010306-8b0c-4a49-8ffa-c99fdfb8adee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cb2e1eb3a5e4c0bab176a8e0be12dc9', 'request_id': 'req-8b010306-8b0c-4a49-8ffa-c99fdfb8adee'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:49.785715 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-8b010306-8b0c-4a49-8ffa-c99fdfb8adee admin admin] The token cache is being invalidate because project 2cb2e1eb3a5e4c0bab176a8e0be12dc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:21:49.785916 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-8b010306-8b0c-4a49-8ffa-c99fdfb8adee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:49.787132 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-8b010306-8b0c-4a49-8ffa-c99fdfb8adee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:49.788992 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 991/1968] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:21:49 2022] DELETE /identity/v3/projects/2cb2e1eb3a5e4c0bab176a8e0be12dc9 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:49.792977 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a48321-94ab-43c2-8ba1-52d0f7e7dc0e service neutron] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:49.805862 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a48321-94ab-43c2-8ba1-52d0f7e7dc0e service neutron] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:49.908535 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da787d3b-bf21-41f2-bf9a-4ff761508969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:49.909992 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da787d3b-bf21-41f2-bf9a-4ff761508969 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:49.909992 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da787d3b-bf21-41f2-bf9a-4ff761508969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:49.910191 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da787d3b-bf21-41f2-bf9a-4ff761508969 admin admin] PATH_INFO: `/v3/users/b5337f7eeef24e34a79ef8c6ba4d027e` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:49.911576 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da787d3b-bf21-41f2-bf9a-4ff761508969 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5337f7eeef24e34a79ef8c6ba4d027e)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:49.914402 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da787d3b-bf21-41f2-bf9a-4ff761508969 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:50.072158 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b010306-8b0c-4a49-8ffa-c99fdfb8adee admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:50.073385 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-8b010306-8b0c-4a49-8ffa-c99fdfb8adee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:50.092723 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b010306-8b0c-4a49-8ffa-c99fdfb8adee admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:50.101142 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421aaee9-5f3f-4a8e-95ad-aaa28de2bec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:50.102186 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421aaee9-5f3f-4a8e-95ad-aaa28de2bec6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:50.102491 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421aaee9-5f3f-4a8e-95ad-aaa28de2bec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:50.102926 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421aaee9-5f3f-4a8e-95ad-aaa28de2bec6 admin admin] PATH_INFO: `/v3/projects/eaf4003b9f254006956f71a1e8ff6d49` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:50.104345 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421aaee9-5f3f-4a8e-95ad-aaa28de2bec6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eaf4003b9f254006956f71a1e8ff6d49)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:50.106311 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421aaee9-5f3f-4a8e-95ad-aaa28de2bec6 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:50.134170 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-da787d3b-bf21-41f2-bf9a-4ff761508969 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5337f7eeef24e34a79ef8c6ba4d027e'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:50.151287 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:50.174013 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-da787d3b-bf21-41f2-bf9a-4ff761508969 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5337f7eeef24e34a79ef8c6ba4d027e'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:50.175015 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-da787d3b-bf21-41f2-bf9a-4ff761508969 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5337f7eeef24e34a79ef8c6ba4d027e'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:50.228010 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-421aaee9-5f3f-4a8e-95ad-aaa28de2bec6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eaf4003b9f254006956f71a1e8ff6d49', 'request_id': 'req-421aaee9-5f3f-4a8e-95ad-aaa28de2bec6'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:50.238100 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-da787d3b-bf21-41f2-bf9a-4ff761508969 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5337f7eeef24e34a79ef8c6ba4d027e'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:50.243081 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-da787d3b-bf21-41f2-bf9a-4ff761508969 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5337f7eeef24e34a79ef8c6ba4d027e'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:50.256837 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 978/1969] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:21:49 2022] DELETE /identity/v3/users/b5337f7eeef24e34a79ef8c6ba4d027e => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:50.335899 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-421aaee9-5f3f-4a8e-95ad-aaa28de2bec6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eaf4003b9f254006956f71a1e8ff6d49', 'request_id': 'req-421aaee9-5f3f-4a8e-95ad-aaa28de2bec6'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:50.342274 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-421aaee9-5f3f-4a8e-95ad-aaa28de2bec6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eaf4003b9f254006956f71a1e8ff6d49', 'request_id': 'req-421aaee9-5f3f-4a8e-95ad-aaa28de2bec6'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:50.355692 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-421aaee9-5f3f-4a8e-95ad-aaa28de2bec6 admin admin] The token cache is being invalidate because project eaf4003b9f254006956f71a1e8ff6d49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:21:50.356087 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-421aaee9-5f3f-4a8e-95ad-aaa28de2bec6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:50.356763 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-421aaee9-5f3f-4a8e-95ad-aaa28de2bec6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:50.358346 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 992/1970] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:21:50 2022] DELETE /identity/v3/projects/eaf4003b9f254006956f71a1e8ff6d49 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:50.362507 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da787d3b-bf21-41f2-bf9a-4ff761508969 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:50.364111 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-da787d3b-bf21-41f2-bf9a-4ff761508969 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:50.433128 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da787d3b-bf21-41f2-bf9a-4ff761508969 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:50.437786 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b9777b-746f-4789-b64e-842a8aeb0947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:50.438663 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b9777b-746f-4789-b64e-842a8aeb0947 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:50.438969 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b9777b-746f-4789-b64e-842a8aeb0947 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:50.439240 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b9777b-746f-4789-b64e-842a8aeb0947 admin admin] PATH_INFO: `/v3/users/6f994c0f7bb044bdb80e14fbbaa64329` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:50.440542 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b9777b-746f-4789-b64e-842a8aeb0947 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f994c0f7bb044bdb80e14fbbaa64329)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:50.442547 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b9777b-746f-4789-b64e-842a8aeb0947 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:50.486262 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421aaee9-5f3f-4a8e-95ad-aaa28de2bec6 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:50.487730 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-421aaee9-5f3f-4a8e-95ad-aaa28de2bec6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:50.558062 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421aaee9-5f3f-4a8e-95ad-aaa28de2bec6 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:50.562778 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890458ca-c2dd-445c-b8cf-1aa4307368ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:50.563682 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890458ca-c2dd-445c-b8cf-1aa4307368ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:50.563960 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890458ca-c2dd-445c-b8cf-1aa4307368ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:50.564235 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890458ca-c2dd-445c-b8cf-1aa4307368ba admin admin] PATH_INFO: `/v3/projects/7118cfff01c34c8abb4aa2144013c485` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:50.565416 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890458ca-c2dd-445c-b8cf-1aa4307368ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=7118cfff01c34c8abb4aa2144013c485)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:50.567390 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890458ca-c2dd-445c-b8cf-1aa4307368ba admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:50.624622 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-54b9777b-746f-4789-b64e-842a8aeb0947 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f994c0f7bb044bdb80e14fbbaa64329'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:50.631596 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-54b9777b-746f-4789-b64e-842a8aeb0947 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f994c0f7bb044bdb80e14fbbaa64329'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:50.632299 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-54b9777b-746f-4789-b64e-842a8aeb0947 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f994c0f7bb044bdb80e14fbbaa64329'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:50.659681 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:50.663362 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-890458ca-c2dd-445c-b8cf-1aa4307368ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7118cfff01c34c8abb4aa2144013c485', 'request_id': 'req-890458ca-c2dd-445c-b8cf-1aa4307368ba'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:50.684549 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-54b9777b-746f-4789-b64e-842a8aeb0947 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f994c0f7bb044bdb80e14fbbaa64329'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:50.685198 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-54b9777b-746f-4789-b64e-842a8aeb0947 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f994c0f7bb044bdb80e14fbbaa64329'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:50.703505 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 979/1971] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:21:50 2022] DELETE /identity/v3/users/6f994c0f7bb044bdb80e14fbbaa64329 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:50.735573 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-890458ca-c2dd-445c-b8cf-1aa4307368ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7118cfff01c34c8abb4aa2144013c485', 'request_id': 'req-890458ca-c2dd-445c-b8cf-1aa4307368ba'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:50.742123 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-890458ca-c2dd-445c-b8cf-1aa4307368ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7118cfff01c34c8abb4aa2144013c485', 'request_id': 'req-890458ca-c2dd-445c-b8cf-1aa4307368ba'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:50.753951 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-890458ca-c2dd-445c-b8cf-1aa4307368ba admin admin] The token cache is being invalidate because project 7118cfff01c34c8abb4aa2144013c485 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:21:50.754248 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-890458ca-c2dd-445c-b8cf-1aa4307368ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:50.755030 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-890458ca-c2dd-445c-b8cf-1aa4307368ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:50.756736 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 993/1972] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:21:50 2022] DELETE /identity/v3/projects/7118cfff01c34c8abb4aa2144013c485 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 02 19:21:50.761919 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c1da64-c0f2-40b4-92a3-a1b5492f4415 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:50.762237 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c1da64-c0f2-40b4-92a3-a1b5492f4415 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:50.762525 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c1da64-c0f2-40b4-92a3-a1b5492f4415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:50.781228 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-57c1da64-c0f2-40b4-92a3-a1b5492f4415 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:50.782462 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-57c1da64-c0f2-40b4-92a3-a1b5492f4415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:50.826002 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:50.844468 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 980/1973] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:21:50.853134 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a618cb8-9c65-4ee7-82d4-11a0ad779433 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:50.853428 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a618cb8-9c65-4ee7-82d4-11a0ad779433 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:50.853708 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a618cb8-9c65-4ee7-82d4-11a0ad779433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:50.872204 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-0a618cb8-9c65-4ee7-82d4-11a0ad779433 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:50.873413 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-0a618cb8-9c65-4ee7-82d4-11a0ad779433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:50.882443 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 994/1974] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:21:50.887484 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c1da64-c0f2-40b4-92a3-a1b5492f4415 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:50.901452 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c1da64-c0f2-40b4-92a3-a1b5492f4415 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:50.906934 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37930153-3664-4e14-a7f4-ce418c533386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:50.907829 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37930153-3664-4e14-a7f4-ce418c533386 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:50.908629 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37930153-3664-4e14-a7f4-ce418c533386 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:50.909055 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37930153-3664-4e14-a7f4-ce418c533386 admin admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:50.909796 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37930153-3664-4e14-a7f4-ce418c533386 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:21:50.910217 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37930153-3664-4e14-a7f4-ce418c533386 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:50.912574 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37930153-3664-4e14-a7f4-ce418c533386 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:50.924794 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 981/1975] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 2 19:21:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:21:50.929988 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a618cb8-9c65-4ee7-82d4-11a0ad779433 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:50.941285 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a618cb8-9c65-4ee7-82d4-11a0ad779433 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:50.947332 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4acada-aaee-4933-a53d-205301165f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:50.948398 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4acada-aaee-4933-a53d-205301165f7b admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:50.948699 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4acada-aaee-4933-a53d-205301165f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:50.949016 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4acada-aaee-4933-a53d-205301165f7b admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:50.949743 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4acada-aaee-4933-a53d-205301165f7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:50.951847 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4acada-aaee-4933-a53d-205301165f7b admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:51.040445 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 995/1976] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:21:50 2022] POST /identity/v3/projects => generated 385 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:21:51.045536 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37930153-3664-4e14-a7f4-ce418c533386 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:51.058306 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37930153-3664-4e14-a7f4-ce418c533386 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:51.120367 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec8a88b-122c-4703-ac4b-1d4c9e17ae25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:51.121239 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec8a88b-122c-4703-ac4b-1d4c9e17ae25 admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:51.121618 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec8a88b-122c-4703-ac4b-1d4c9e17ae25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:51.122006 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec8a88b-122c-4703-ac4b-1d4c9e17ae25 admin admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:51.123279 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec8a88b-122c-4703-ac4b-1d4c9e17ae25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:51.127075 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec8a88b-122c-4703-ac4b-1d4c9e17ae25 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:51.142731 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4acada-aaee-4933-a53d-205301165f7b admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:51.144559 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-9d4acada-aaee-4933-a53d-205301165f7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:51.159376 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4acada-aaee-4933-a53d-205301165f7b admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:51.173642 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9d8327-7c8e-4dc6-a89a-3ad1dbb81087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:51.174586 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9d8327-7c8e-4dc6-a89a-3ad1dbb81087 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:51.174913 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9d8327-7c8e-4dc6-a89a-3ad1dbb81087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:51.175196 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9d8327-7c8e-4dc6-a89a-3ad1dbb81087 admin admin] PATH_INFO: `/v3/projects/a412e0e750c640909f6908d3f8d8dad6` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:51.176478 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9d8327-7c8e-4dc6-a89a-3ad1dbb81087 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a412e0e750c640909f6908d3f8d8dad6)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:51.178406 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9d8327-7c8e-4dc6-a89a-3ad1dbb81087 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:51.226517 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 982/1977] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:21:51 2022] POST /identity/v3/users => generated 293 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 02 19:21:51.230980 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec8a88b-122c-4703-ac4b-1d4c9e17ae25 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:51.245107 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec8a88b-122c-4703-ac4b-1d4c9e17ae25 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:51.250449 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b77480e-bf3b-4447-ac63-cce0108d043d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:51.251186 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b77480e-bf3b-4447-ac63-cce0108d043d admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:51.251469 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b77480e-bf3b-4447-ac63-cce0108d043d admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:51.251735 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b77480e-bf3b-4447-ac63-cce0108d043d admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:51.252468 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b77480e-bf3b-4447-ac63-cce0108d043d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:51.254076 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b77480e-bf3b-4447-ac63-cce0108d043d admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:51.268289 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 983/1978] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:21:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:21:51.270064 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b77480e-bf3b-4447-ac63-cce0108d043d admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:51.282489 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b77480e-bf3b-4447-ac63-cce0108d043d admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:51.285218 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-3d9d8327-7c8e-4dc6-a89a-3ad1dbb81087 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a412e0e750c640909f6908d3f8d8dad6', 'request_id': 'req-3d9d8327-7c8e-4dc6-a89a-3ad1dbb81087'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:51.341809 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c367cd22-85e2-45fe-b8a8-914dcbe12dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:51.342625 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c367cd22-85e2-45fe-b8a8-914dcbe12dfe admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:51.342776 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c367cd22-85e2-45fe-b8a8-914dcbe12dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:51.343056 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c367cd22-85e2-45fe-b8a8-914dcbe12dfe admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:51.343780 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c367cd22-85e2-45fe-b8a8-914dcbe12dfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:51.345368 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c367cd22-85e2-45fe-b8a8-914dcbe12dfe admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:51.355009 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 984/1979] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:21:51 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:21:51.358607 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-3d9d8327-7c8e-4dc6-a89a-3ad1dbb81087 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a412e0e750c640909f6908d3f8d8dad6', 'request_id': 'req-3d9d8327-7c8e-4dc6-a89a-3ad1dbb81087'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:51.358975 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c367cd22-85e2-45fe-b8a8-914dcbe12dfe admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:51.365393 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-3d9d8327-7c8e-4dc6-a89a-3ad1dbb81087 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a412e0e750c640909f6908d3f8d8dad6', 'request_id': 'req-3d9d8327-7c8e-4dc6-a89a-3ad1dbb81087'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:51.370026 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c367cd22-85e2-45fe-b8a8-914dcbe12dfe admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:51.374818 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cc6cc7-cb89-492e-9382-e5d43aae18ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:51.375726 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc6cc7-cb89-492e-9382-e5d43aae18ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:51.377078 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc6cc7-cb89-492e-9382-e5d43aae18ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:51.377078 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc6cc7-cb89-492e-9382-e5d43aae18ab admin admin] PATH_INFO: `/v3/projects/038af064a17f42b9be1aa0a23567503f/users/595e7c5c699d48818775e8f2ae5ed3a9/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:51.377533 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-3d9d8327-7c8e-4dc6-a89a-3ad1dbb81087 admin admin] The token cache is being invalidate because project a412e0e750c640909f6908d3f8d8dad6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:21:51.377819 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-3d9d8327-7c8e-4dc6-a89a-3ad1dbb81087 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:51.378485 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-3d9d8327-7c8e-4dc6-a89a-3ad1dbb81087 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:51.380106 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 996/1980] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:21:51 2022] DELETE /identity/v3/projects/a412e0e750c640909f6908d3f8d8dad6 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 02 19:21:51.385322 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1cd5a2-f59d-4013-9e27-851a8ec89cd0 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:51.385611 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1cd5a2-f59d-4013-9e27-851a8ec89cd0 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:51.386140 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1cd5a2-f59d-4013-9e27-851a8ec89cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:51.391587 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cc6cc7-cb89-492e-9382-e5d43aae18ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=038af064a17f42b9be1aa0a23567503f, user_id=595e7c5c699d48818775e8f2ae5ed3a9, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:51.394208 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cc6cc7-cb89-492e-9382-e5d43aae18ab admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:51.403598 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-8a1cd5a2-f59d-4013-9e27-851a8ec89cd0 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:51.404798 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-8a1cd5a2-f59d-4013-9e27-851a8ec89cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:51.412821 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 997/1981] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 19:21:51.420779 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0202c170-cffa-470b-b35e-d71809f797ee None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:51.421067 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0202c170-cffa-470b-b35e-d71809f797ee None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:51.421380 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0202c170-cffa-470b-b35e-d71809f797ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:51.439919 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-0202c170-cffa-470b-b35e-d71809f797ee None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:51.441105 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-0202c170-cffa-470b-b35e-d71809f797ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:51.454233 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 998/1982] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 19:21:51.457577 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 985/1983] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:21:51 2022] PUT /identity/v3/projects/038af064a17f42b9be1aa0a23567503f/users/595e7c5c699d48818775e8f2ae5ed3a9/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 02 19:21:51.458511 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0202c170-cffa-470b-b35e-d71809f797ee None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:51.464681 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34461774-0998-4ec9-bbdb-f47ebcd24499 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:51.464971 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34461774-0998-4ec9-bbdb-f47ebcd24499 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:51.465248 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34461774-0998-4ec9-bbdb-f47ebcd24499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:51.468079 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0202c170-cffa-470b-b35e-d71809f797ee None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:51.496466 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-34461774-0998-4ec9-bbdb-f47ebcd24499 None None] MFA Rules not processed for user `595e7c5c699d48818775e8f2ae5ed3a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:51.497975 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-34461774-0998-4ec9-bbdb-f47ebcd24499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:51.511063 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:51.527263 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d746810b-210f-40fb-a941-0820b2d6c61d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:51.527975 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d746810b-210f-40fb-a941-0820b2d6c61d admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:51.528264 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d746810b-210f-40fb-a941-0820b2d6c61d admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:51.528544 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d746810b-210f-40fb-a941-0820b2d6c61d admin admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:51.529292 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d746810b-210f-40fb-a941-0820b2d6c61d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:21:51.529603 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d746810b-210f-40fb-a941-0820b2d6c61d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:51.531293 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d746810b-210f-40fb-a941-0820b2d6c61d admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:51.543900 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 999/1984] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 2 19:21:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:21:51.548216 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d746810b-210f-40fb-a941-0820b2d6c61d admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:51.559125 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d746810b-210f-40fb-a941-0820b2d6c61d admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:51.565952 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db56234-c7c0-4ac3-a5d5-b0b0dee1d1db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:51.566727 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db56234-c7c0-4ac3-a5d5-b0b0dee1d1db admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:51.567047 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db56234-c7c0-4ac3-a5d5-b0b0dee1d1db admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:51.567331 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db56234-c7c0-4ac3-a5d5-b0b0dee1d1db admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:51.567836 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 986/1985] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 19:21:51.568593 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db56234-c7c0-4ac3-a5d5-b0b0dee1d1db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:51.570580 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db56234-c7c0-4ac3-a5d5-b0b0dee1d1db admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:51.577167 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:DELETE=11,keystone:SELECT=86,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:51.579664 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34461774-0998-4ec9-bbdb-f47ebcd24499 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:51.581191 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-34461774-0998-4ec9-bbdb-f47ebcd24499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:51.643795 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34461774-0998-4ec9-bbdb-f47ebcd24499 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:51.647597 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3680aa60-6431-4338-bfb6-f56c90593a09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:51.648234 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3680aa60-6431-4338-bfb6-f56c90593a09 service cinder] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:51.648519 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3680aa60-6431-4338-bfb6-f56c90593a09 service cinder] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:51.648796 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3680aa60-6431-4338-bfb6-f56c90593a09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:51.657877 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3680aa60-6431-4338-bfb6-f56c90593a09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:51.659355 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3680aa60-6431-4338-bfb6-f56c90593a09 service cinder] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:51.668568 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 987/1986] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:21:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 02 19:21:51.683079 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1000/1987] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:21:51 2022] POST /identity/v3/projects => generated 373 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 02 19:21:51.687135 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3680aa60-6431-4338-bfb6-f56c90593a09 service cinder] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:51.693555 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db56234-c7c0-4ac3-a5d5-b0b0dee1d1db admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:51.695430 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-2db56234-c7c0-4ac3-a5d5-b0b0dee1d1db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:51.699497 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3680aa60-6431-4338-bfb6-f56c90593a09 service cinder] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:51.762949 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dc2b9d-9111-4e5e-8937-2f82b4b9b344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:51.763667 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dc2b9d-9111-4e5e-8937-2f82b4b9b344 admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:51.763952 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dc2b9d-9111-4e5e-8937-2f82b4b9b344 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:51.764228 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dc2b9d-9111-4e5e-8937-2f82b4b9b344 admin admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:51.764954 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dc2b9d-9111-4e5e-8937-2f82b4b9b344 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:51.765285 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db56234-c7c0-4ac3-a5d5-b0b0dee1d1db admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:51.766912 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dc2b9d-9111-4e5e-8937-2f82b4b9b344 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:51.771337 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bbd3dd-f62a-45b0-81c3-aa9e0e4ee300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:51.772223 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bbd3dd-f62a-45b0-81c3-aa9e0e4ee300 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:51.772503 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bbd3dd-f62a-45b0-81c3-aa9e0e4ee300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:51.772778 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bbd3dd-f62a-45b0-81c3-aa9e0e4ee300 admin admin] PATH_INFO: `/v3/users/595e7c5c699d48818775e8f2ae5ed3a9` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:51.774003 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bbd3dd-f62a-45b0-81c3-aa9e0e4ee300 admin admin] RBAC: Authorizing `identity:delete_user(user_id=595e7c5c699d48818775e8f2ae5ed3a9)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:51.775971 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bbd3dd-f62a-45b0-81c3-aa9e0e4ee300 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:51.828439 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 988/1988] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:21:51 2022] POST /identity/v3/users => generated 287 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 02 19:21:51.832329 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dc2b9d-9111-4e5e-8937-2f82b4b9b344 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:51.843064 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dc2b9d-9111-4e5e-8937-2f82b4b9b344 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:51.847929 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f21f6db-d469-4b5e-bcb7-68f420ca4710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:51.848605 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f21f6db-d469-4b5e-bcb7-68f420ca4710 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:51.848896 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f21f6db-d469-4b5e-bcb7-68f420ca4710 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:51.849791 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f21f6db-d469-4b5e-bcb7-68f420ca4710 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:51.850526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f21f6db-d469-4b5e-bcb7-68f420ca4710 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:51.852164 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f21f6db-d469-4b5e-bcb7-68f420ca4710 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:51.862638 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 989/1989] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:21:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 02 19:21:51.867765 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f21f6db-d469-4b5e-bcb7-68f420ca4710 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:51.877916 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f21f6db-d469-4b5e-bcb7-68f420ca4710 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:51.882727 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d7d3f5-87c0-47f8-91e4-294ad3a2d682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:51.883423 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d7d3f5-87c0-47f8-91e4-294ad3a2d682 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:51.883713 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d7d3f5-87c0-47f8-91e4-294ad3a2d682 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:51.883985 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d7d3f5-87c0-47f8-91e4-294ad3a2d682 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:51.884737 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d7d3f5-87c0-47f8-91e4-294ad3a2d682 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:51.886346 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d7d3f5-87c0-47f8-91e4-294ad3a2d682 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:51.897228 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 990/1990] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:21:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 02 19:21:51.902179 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d7d3f5-87c0-47f8-91e4-294ad3a2d682 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:51.912900 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d7d3f5-87c0-47f8-91e4-294ad3a2d682 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:51.917707 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075dbfcc-236e-4ad3-a94f-903774a31a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:51.918818 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075dbfcc-236e-4ad3-a94f-903774a31a8a admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:51.919110 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075dbfcc-236e-4ad3-a94f-903774a31a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:51.919393 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075dbfcc-236e-4ad3-a94f-903774a31a8a admin admin] PATH_INFO: `/v3/projects/8c71f8729510465b9868464021c28d3c/users/ea59c4e399f14500b5d85178acc8a075/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:51.935270 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075dbfcc-236e-4ad3-a94f-903774a31a8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c71f8729510465b9868464021c28d3c, user_id=ea59c4e399f14500b5d85178acc8a075, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:51.935477 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-07bbd3dd-f62a-45b0-81c3-aa9e0e4ee300 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '595e7c5c699d48818775e8f2ae5ed3a9'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:51.937941 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075dbfcc-236e-4ad3-a94f-903774a31a8a admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:51.943131 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-07bbd3dd-f62a-45b0-81c3-aa9e0e4ee300 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '595e7c5c699d48818775e8f2ae5ed3a9'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:52.017938 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:52.022840 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 991/1991] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:21:51 2022] PUT /identity/v3/projects/8c71f8729510465b9868464021c28d3c/users/ea59c4e399f14500b5d85178acc8a075/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 02 19:21:52.034654 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5291d0-609f-46c1-a9a8-aade3860e082 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:52.035050 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5291d0-609f-46c1-a9a8-aade3860e082 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:52.035336 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5291d0-609f-46c1-a9a8-aade3860e082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:52.044668 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-07bbd3dd-f62a-45b0-81c3-aa9e0e4ee300 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '595e7c5c699d48818775e8f2ae5ed3a9'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:52.045925 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-07bbd3dd-f62a-45b0-81c3-aa9e0e4ee300 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '595e7c5c699d48818775e8f2ae5ed3a9'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:52.046590 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-07bbd3dd-f62a-45b0-81c3-aa9e0e4ee300 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '595e7c5c699d48818775e8f2ae5ed3a9'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:52.063685 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-ef5291d0-609f-46c1-a9a8-aade3860e082 None None] MFA Rules not processed for user `ea59c4e399f14500b5d85178acc8a075`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:52.064916 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-ef5291d0-609f-46c1-a9a8-aade3860e082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:52.067819 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1001/1992] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:21:51 2022] DELETE /identity/v3/users/595e7c5c699d48818775e8f2ae5ed3a9 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 02 19:21:52.072019 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf8aaa7-ccf2-4d9a-bf71-6b8582c402ac None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:52.072315 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf8aaa7-ccf2-4d9a-bf71-6b8582c402ac None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:52.072596 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf8aaa7-ccf2-4d9a-bf71-6b8582c402ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:52.090076 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-6bf8aaa7-ccf2-4d9a-bf71-6b8582c402ac None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:52.091317 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-6bf8aaa7-ccf2-4d9a-bf71-6b8582c402ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:52.140822 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 992/1993] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:52 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 19:21:52.144032 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5291d0-609f-46c1-a9a8-aade3860e082 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:52.145407 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-ef5291d0-609f-46c1-a9a8-aade3860e082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:52.169985 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1002/1994] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 19:21:52.179701 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf8aaa7-ccf2-4d9a-bf71-6b8582c402ac None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:52.182801 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-6bf8aaa7-ccf2-4d9a-bf71-6b8582c402ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:52.190807 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:52.222359 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5291d0-609f-46c1-a9a8-aade3860e082 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:52.229505 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c30aae5-f7f5-4db1-b0a8-8d08a4f551ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:52.229505 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c30aae5-f7f5-4db1-b0a8-8d08a4f551ca admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:52.229505 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c30aae5-f7f5-4db1-b0a8-8d08a4f551ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:52.229505 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c30aae5-f7f5-4db1-b0a8-8d08a4f551ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:52.229992 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c30aae5-f7f5-4db1-b0a8-8d08a4f551ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:52.231991 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c30aae5-f7f5-4db1-b0a8-8d08a4f551ca admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:52.246657 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf8aaa7-ccf2-4d9a-bf71-6b8582c402ac None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:52.252531 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82fa8c7-446e-4ba9-8729-7f1710f5431a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:52.261487 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82fa8c7-446e-4ba9-8729-7f1710f5431a service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:52.261487 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82fa8c7-446e-4ba9-8729-7f1710f5431a service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:52.261487 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82fa8c7-446e-4ba9-8729-7f1710f5431a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:52.264790 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82fa8c7-446e-4ba9-8729-7f1710f5431a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:52.266264 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82fa8c7-446e-4ba9-8729-7f1710f5431a service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:52.277535 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1003/1995] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:21:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 02 19:21:52.335213 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 993/1996] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:21:52 2022] POST /identity/v3/projects => generated 375 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:21:52.336510 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82fa8c7-446e-4ba9-8729-7f1710f5431a service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:52.350557 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82fa8c7-446e-4ba9-8729-7f1710f5431a service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:52.420323 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407b4177-8816-4a5e-a0e5-cd68cfd892e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:52.421439 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b4177-8816-4a5e-a0e5-cd68cfd892e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:52.421886 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b4177-8816-4a5e-a0e5-cd68cfd892e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:52.422283 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b4177-8816-4a5e-a0e5-cd68cfd892e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:52.423546 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407b4177-8816-4a5e-a0e5-cd68cfd892e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:52.425660 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407b4177-8816-4a5e-a0e5-cd68cfd892e7 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:52.535083 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1004/1997] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:21:52 2022] POST /identity/v3/users => generated 288 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:21:52.537092 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c30aae5-f7f5-4db1-b0a8-8d08a4f551ca admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:52.549829 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c30aae5-f7f5-4db1-b0a8-8d08a4f551ca admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:52.556500 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b344783-9a8f-4af3-a5d0-007f16a0c343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:52.557302 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b344783-9a8f-4af3-a5d0-007f16a0c343 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:52.557946 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b344783-9a8f-4af3-a5d0-007f16a0c343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:52.557946 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b344783-9a8f-4af3-a5d0-007f16a0c343 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:52.558784 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b344783-9a8f-4af3-a5d0-007f16a0c343 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:52.561012 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b344783-9a8f-4af3-a5d0-007f16a0c343 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:52.573244 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 994/1998] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:21:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:21:52.577672 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407b4177-8816-4a5e-a0e5-cd68cfd892e7 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:52.595392 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407b4177-8816-4a5e-a0e5-cd68cfd892e7 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:52.602554 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a28bf8-2613-4705-bac5-07e2bfd46c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:52.604295 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a28bf8-2613-4705-bac5-07e2bfd46c74 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:52.605118 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a28bf8-2613-4705-bac5-07e2bfd46c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:52.605540 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a28bf8-2613-4705-bac5-07e2bfd46c74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:52.609039 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a28bf8-2613-4705-bac5-07e2bfd46c74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:52.611172 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a28bf8-2613-4705-bac5-07e2bfd46c74 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:52.624766 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1005/1999] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:21:52 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:21:52.629573 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b344783-9a8f-4af3-a5d0-007f16a0c343 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:52.641544 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b344783-9a8f-4af3-a5d0-007f16a0c343 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:52.647150 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e46cc4-1492-46cc-b5f6-7e62459746bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:52.648225 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e46cc4-1492-46cc-b5f6-7e62459746bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:52.648765 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e46cc4-1492-46cc-b5f6-7e62459746bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:52.649060 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e46cc4-1492-46cc-b5f6-7e62459746bc admin admin] PATH_INFO: `/v3/projects/f90fcb16902e4d899c254718007ba426/users/9e43e8fa3b1747b3942cec0dd05e882b/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:52.667542 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e46cc4-1492-46cc-b5f6-7e62459746bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f90fcb16902e4d899c254718007ba426, user_id=9e43e8fa3b1747b3942cec0dd05e882b, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:52.670326 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e46cc4-1492-46cc-b5f6-7e62459746bc admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:52.691744 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a28bf8-2613-4705-bac5-07e2bfd46c74 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:52.694260 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-87a28bf8-2613-4705-bac5-07e2bfd46c74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:52.709052 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a28bf8-2613-4705-bac5-07e2bfd46c74 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:52.714479 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb9cee0-258d-4395-abb1-f33eec9aafb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:52.715208 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb9cee0-258d-4395-abb1-f33eec9aafb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:52.720389 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb9cee0-258d-4395-abb1-f33eec9aafb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:52.720389 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb9cee0-258d-4395-abb1-f33eec9aafb6 admin admin] PATH_INFO: `/v3/projects/038af064a17f42b9be1aa0a23567503f` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:52.720389 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb9cee0-258d-4395-abb1-f33eec9aafb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=038af064a17f42b9be1aa0a23567503f)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:52.720389 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb9cee0-258d-4395-abb1-f33eec9aafb6 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:52.744346 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 995/2000] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:21:52 2022] PUT /identity/v3/projects/f90fcb16902e4d899c254718007ba426/users/9e43e8fa3b1747b3942cec0dd05e882b/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:52.757075 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e163b-6fdd-4198-9051-7ff5a37475a7 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:52.757759 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e163b-6fdd-4198-9051-7ff5a37475a7 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:52.757759 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e163b-6fdd-4198-9051-7ff5a37475a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:52.789984 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-db0e163b-6fdd-4198-9051-7ff5a37475a7 None None] MFA Rules not processed for user `9e43e8fa3b1747b3942cec0dd05e882b`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:52.791188 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-db0e163b-6fdd-4198-9051-7ff5a37475a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:52.821777 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:52.861609 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 996/2001] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:21:52.867309 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0e163b-6fdd-4198-9051-7ff5a37475a7 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:52.874664 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-4bb9cee0-258d-4395-abb1-f33eec9aafb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '038af064a17f42b9be1aa0a23567503f', 'request_id': 'req-4bb9cee0-258d-4395-abb1-f33eec9aafb6'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:52.937556 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0e163b-6fdd-4198-9051-7ff5a37475a7 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:52.953074 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-4bb9cee0-258d-4395-abb1-f33eec9aafb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '038af064a17f42b9be1aa0a23567503f', 'request_id': 'req-4bb9cee0-258d-4395-abb1-f33eec9aafb6'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:52.960207 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-4bb9cee0-258d-4395-abb1-f33eec9aafb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '038af064a17f42b9be1aa0a23567503f', 'request_id': 'req-4bb9cee0-258d-4395-abb1-f33eec9aafb6'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:52.973710 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-4bb9cee0-258d-4395-abb1-f33eec9aafb6 admin admin] The token cache is being invalidate because project 038af064a17f42b9be1aa0a23567503f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:21:52.974054 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-4bb9cee0-258d-4395-abb1-f33eec9aafb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:52.974710 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-4bb9cee0-258d-4395-abb1-f33eec9aafb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:52.976379 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1006/2002] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:21:52 2022] DELETE /identity/v3/projects/038af064a17f42b9be1aa0a23567503f => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 02 19:21:52.981458 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b29b26-b662-4a6c-9899-9d1c3a19d07e None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:52.981848 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b29b26-b662-4a6c-9899-9d1c3a19d07e None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:52.982150 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b29b26-b662-4a6c-9899-9d1c3a19d07e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:53.001982 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c676f4e2-2b25-4dc0-ae5b-237e9a6ed0b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:53.002378 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-d6b29b26-b662-4a6c-9899-9d1c3a19d07e None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:53.003272 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c676f4e2-2b25-4dc0-ae5b-237e9a6ed0b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:53.003640 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-d6b29b26-b662-4a6c-9899-9d1c3a19d07e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:53.003977 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c676f4e2-2b25-4dc0-ae5b-237e9a6ed0b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:53.004437 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c676f4e2-2b25-4dc0-ae5b-237e9a6ed0b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:53.005291 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c676f4e2-2b25-4dc0-ae5b-237e9a6ed0b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:53.008377 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c676f4e2-2b25-4dc0-ae5b-237e9a6ed0b6 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:53.012661 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1007/2003] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 19:21:53.022479 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d4897a-4a43-4375-928d-fb1125160516 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:53.022952 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d4897a-4a43-4375-928d-fb1125160516 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:53.023248 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d4897a-4a43-4375-928d-fb1125160516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:53.032453 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:53.057242 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-e9d4897a-4a43-4375-928d-fb1125160516 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:53.057242 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-e9d4897a-4a43-4375-928d-fb1125160516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:53.062104 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1008/2004] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 19:21:53.063215 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d4897a-4a43-4375-928d-fb1125160516 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:53.075168 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d4897a-4a43-4375-928d-fb1125160516 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:53.078463 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68e77a3-d793-48ed-a824-5ce3aa320c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:53.079182 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68e77a3-d793-48ed-a824-5ce3aa320c45 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:53.079471 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68e77a3-d793-48ed-a824-5ce3aa320c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:53.079750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68e77a3-d793-48ed-a824-5ce3aa320c45 admin admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:53.080485 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68e77a3-d793-48ed-a824-5ce3aa320c45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:21:53.080795 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68e77a3-d793-48ed-a824-5ce3aa320c45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:53.082553 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68e77a3-d793-48ed-a824-5ce3aa320c45 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:53.107288 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 997/2005] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:21:52 2022] POST /identity/v3/projects => generated 375 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 02 19:21:53.112036 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c676f4e2-2b25-4dc0-ae5b-237e9a6ed0b6 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:53.114262 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-c676f4e2-2b25-4dc0-ae5b-237e9a6ed0b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:53.128647 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1009/2006] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 2 19:21:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:21:53.134145 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68e77a3-d793-48ed-a824-5ce3aa320c45 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:53.144532 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68e77a3-d793-48ed-a824-5ce3aa320c45 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:53.205209 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c676f4e2-2b25-4dc0-ae5b-237e9a6ed0b6 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:53.211303 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b42845-9988-4163-82d4-da2508f3a31c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:53.212225 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b42845-9988-4163-82d4-da2508f3a31c admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:53.212653 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b42845-9988-4163-82d4-da2508f3a31c admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:53.212938 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b42845-9988-4163-82d4-da2508f3a31c admin admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:53.213650 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b42845-9988-4163-82d4-da2508f3a31c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:53.216497 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b42845-9988-4163-82d4-da2508f3a31c admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:53.222988 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f43b4f0-c7af-4b1a-8a1b-bdac7925f9b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:53.223560 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f43b4f0-c7af-4b1a-8a1b-bdac7925f9b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:53.223849 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f43b4f0-c7af-4b1a-8a1b-bdac7925f9b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:53.224655 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f43b4f0-c7af-4b1a-8a1b-bdac7925f9b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:53.225356 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f43b4f0-c7af-4b1a-8a1b-bdac7925f9b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:53.227635 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:53.227786 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f43b4f0-c7af-4b1a-8a1b-bdac7925f9b9 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:53.328498 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 998/2007] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:21:53 2022] POST /identity/v3/users => generated 294 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:21:53.330718 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b42845-9988-4163-82d4-da2508f3a31c admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:53.331771 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1010/2008] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:21:53 2022] POST /identity/v3/projects => generated 359 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 02 19:21:53.336981 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f43b4f0-c7af-4b1a-8a1b-bdac7925f9b9 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:53.351235 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b42845-9988-4163-82d4-da2508f3a31c admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:53.440977 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c93600f-8569-4d8b-ba35-3f89d04122f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:53.441689 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c93600f-8569-4d8b-ba35-3f89d04122f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:53.441977 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c93600f-8569-4d8b-ba35-3f89d04122f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:53.442254 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c93600f-8569-4d8b-ba35-3f89d04122f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:53.443104 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c93600f-8569-4d8b-ba35-3f89d04122f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:53.444989 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c93600f-8569-4d8b-ba35-3f89d04122f4 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:53.447005 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f43b4f0-c7af-4b1a-8a1b-bdac7925f9b9 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:53.455358 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7067e2bf-2d61-4e17-97d8-fd85d183c23c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:53.456323 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7067e2bf-2d61-4e17-97d8-fd85d183c23c admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:53.456793 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7067e2bf-2d61-4e17-97d8-fd85d183c23c admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:53.456900 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 999/2009] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:21:53 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:21:53.457458 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7067e2bf-2d61-4e17-97d8-fd85d183c23c admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:53.457843 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c93600f-8569-4d8b-ba35-3f89d04122f4 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:53.458614 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7067e2bf-2d61-4e17-97d8-fd85d183c23c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:53.459761 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-6c93600f-8569-4d8b-ba35-3f89d04122f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:53.461420 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7067e2bf-2d61-4e17-97d8-fd85d183c23c admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:53.475086 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c93600f-8569-4d8b-ba35-3f89d04122f4 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:53.481736 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7286fe3-785f-4da6-9a6f-ef0ba3a663c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:53.487070 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7286fe3-785f-4da6-9a6f-ef0ba3a663c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:53.487070 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7286fe3-785f-4da6-9a6f-ef0ba3a663c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:53.487070 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7286fe3-785f-4da6-9a6f-ef0ba3a663c0 admin admin] PATH_INFO: `/v3/users/4e541a37b195462187d9ce3eb4073c38` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:53.487070 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7286fe3-785f-4da6-9a6f-ef0ba3a663c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e541a37b195462187d9ce3eb4073c38)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:53.487691 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7286fe3-785f-4da6-9a6f-ef0ba3a663c0 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:53.623650 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1011/2010] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:21:53 2022] POST /identity/v3/users => generated 280 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:21:53.625191 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7067e2bf-2d61-4e17-97d8-fd85d183c23c admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:53.636780 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7067e2bf-2d61-4e17-97d8-fd85d183c23c admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:53.641871 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee65c37d-30bd-4b3a-99d9-9b8cf25142e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:53.642801 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee65c37d-30bd-4b3a-99d9-9b8cf25142e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:53.643088 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee65c37d-30bd-4b3a-99d9-9b8cf25142e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:53.643520 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee65c37d-30bd-4b3a-99d9-9b8cf25142e9 admin admin] PATH_INFO: `/v3/projects/5b70b7aa4377447b897f75e5dca91174/users/b4dd78d14b4144b2b4245f16480459aa/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:53.662454 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee65c37d-30bd-4b3a-99d9-9b8cf25142e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b70b7aa4377447b897f75e5dca91174, user_id=b4dd78d14b4144b2b4245f16480459aa, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:53.665873 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee65c37d-30bd-4b3a-99d9-9b8cf25142e9 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:53.778249 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1012/2011] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:21:53 2022] PUT /identity/v3/projects/5b70b7aa4377447b897f75e5dca91174/users/b4dd78d14b4144b2b4245f16480459aa/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:53.779236 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee65c37d-30bd-4b3a-99d9-9b8cf25142e9 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:53.825515 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-f7286fe3-785f-4da6-9a6f-ef0ba3a663c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e541a37b195462187d9ce3eb4073c38'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:53.836633 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-f7286fe3-785f-4da6-9a6f-ef0ba3a663c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e541a37b195462187d9ce3eb4073c38'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:53.838443 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-f7286fe3-785f-4da6-9a6f-ef0ba3a663c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e541a37b195462187d9ce3eb4073c38'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:53.854528 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:53.879883 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee65c37d-30bd-4b3a-99d9-9b8cf25142e9 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:53.884634 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e430a486-c7e3-4dc8-b3d9-ce6d63053c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:53.885366 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e430a486-c7e3-4dc8-b3d9-ce6d63053c44 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:53.885794 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e430a486-c7e3-4dc8-b3d9-ce6d63053c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:53.885916 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e430a486-c7e3-4dc8-b3d9-ce6d63053c44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:53.886874 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e430a486-c7e3-4dc8-b3d9-ce6d63053c44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:53.889392 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e430a486-c7e3-4dc8-b3d9-ce6d63053c44 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:53.900861 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1013/2012] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:21:53 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:21:53.901656 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e430a486-c7e3-4dc8-b3d9-ce6d63053c44 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:53.904042 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-e430a486-c7e3-4dc8-b3d9-ce6d63053c44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:53.918640 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e430a486-c7e3-4dc8-b3d9-ce6d63053c44 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:53.923572 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8285a9-b84f-47c3-9295-f1a7db55b96a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:53.924250 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8285a9-b84f-47c3-9295-f1a7db55b96a admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:53.924531 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8285a9-b84f-47c3-9295-f1a7db55b96a admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:53.924810 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8285a9-b84f-47c3-9295-f1a7db55b96a admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:53.925549 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8285a9-b84f-47c3-9295-f1a7db55b96a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:53.927218 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8285a9-b84f-47c3-9295-f1a7db55b96a admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:53.929921 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-f7286fe3-785f-4da6-9a6f-ef0ba3a663c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e541a37b195462187d9ce3eb4073c38'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:53.930493 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-f7286fe3-785f-4da6-9a6f-ef0ba3a663c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e541a37b195462187d9ce3eb4073c38'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:53.946920 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1014/2013] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:21:53 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:21:53.946920 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8285a9-b84f-47c3-9295-f1a7db55b96a admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:53.949974 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-db8285a9-b84f-47c3-9295-f1a7db55b96a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:53.959332 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1000/2014] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:21:53 2022] DELETE /identity/v3/users/4e541a37b195462187d9ce3eb4073c38 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:53.961074 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7286fe3-785f-4da6-9a6f-ef0ba3a663c0 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:53.967822 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8285a9-b84f-47c3-9295-f1a7db55b96a admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:54.053531 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7286fe3-785f-4da6-9a6f-ef0ba3a663c0 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:54.054984 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbe5eec-65cf-4baf-a720-8d6afa3329e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:54.055788 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbe5eec-65cf-4baf-a720-8d6afa3329e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:54.056208 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbe5eec-65cf-4baf-a720-8d6afa3329e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:54.056598 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbe5eec-65cf-4baf-a720-8d6afa3329e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:54.057640 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbe5eec-65cf-4baf-a720-8d6afa3329e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:54.059407 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbe5eec-65cf-4baf-a720-8d6afa3329e8 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:54.059598 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27c4930-e0f9-4015-8fe3-e6df25c19660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:54.060760 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27c4930-e0f9-4015-8fe3-e6df25c19660 admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:54.061044 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27c4930-e0f9-4015-8fe3-e6df25c19660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:54.061322 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27c4930-e0f9-4015-8fe3-e6df25c19660 admin admin] PATH_INFO: `/v3/projects/5b70b7aa4377447b897f75e5dca91174/users/b4dd78d14b4144b2b4245f16480459aa/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:54.064035 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27c4930-e0f9-4015-8fe3-e6df25c19660 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b70b7aa4377447b897f75e5dca91174, user_id=b4dd78d14b4144b2b4245f16480459aa, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:54.067199 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27c4930-e0f9-4015-8fe3-e6df25c19660 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:54.070985 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:54.074805 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1015/2015] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:21:53 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:21:54.074805 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10702401-990e-4228-9eab-1b3b840174e0 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:54.074805 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10702401-990e-4228-9eab-1b3b840174e0 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:54.074805 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10702401-990e-4228-9eab-1b3b840174e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:54.085000 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1001/2016] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:21:53 2022] PUT /identity/v3/projects/5b70b7aa4377447b897f75e5dca91174/users/b4dd78d14b4144b2b4245f16480459aa/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:54.086129 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27c4930-e0f9-4015-8fe3-e6df25c19660 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:54.101381 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-10702401-990e-4228-9eab-1b3b840174e0 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:54.102690 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-10702401-990e-4228-9eab-1b3b840174e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:54.173741 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27c4930-e0f9-4015-8fe3-e6df25c19660 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:54.180975 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1016/2017] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:21:54.180975 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d5d05-5036-4b01-9968-3ab8cc6c45bb None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:54.180975 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d5d05-5036-4b01-9968-3ab8cc6c45bb None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:54.180975 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d5d05-5036-4b01-9968-3ab8cc6c45bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:54.187396 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2c425c-f69a-453f-b484-381d578fd358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:54.189197 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c425c-f69a-453f-b484-381d578fd358 admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:54.189491 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c425c-f69a-453f-b484-381d578fd358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:54.189778 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c425c-f69a-453f-b484-381d578fd358 admin admin] PATH_INFO: `/v3/projects/462d3a4a6a4249d980ff6bbb77ee5dee/users/e58f8aa100724d448846f3c1722d66cb/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:54.210774 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2c425c-f69a-453f-b484-381d578fd358 admin admin] RBAC: Authorizing `identity:create_grant(project_id=462d3a4a6a4249d980ff6bbb77ee5dee, user_id=e58f8aa100724d448846f3c1722d66cb, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:54.215350 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2c425c-f69a-453f-b484-381d578fd358 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:54.218068 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-ec8d5d05-5036-4b01-9968-3ab8cc6c45bb None None] MFA Rules not processed for user `b4dd78d14b4144b2b4245f16480459aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:54.219438 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-ec8d5d05-5036-4b01-9968-3ab8cc6c45bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:54.271702 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:54.279715 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1002/2018] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:21:54 2022] PUT /identity/v3/projects/462d3a4a6a4249d980ff6bbb77ee5dee/users/e58f8aa100724d448846f3c1722d66cb/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:54.280770 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2c425c-f69a-453f-b484-381d578fd358 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:54.288680 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-6f2c425c-f69a-453f-b484-381d578fd358 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:54.304346 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1017/2019] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:54 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:21:54.306123 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a0b8f4-228f-42b2-8038-ee7d2c420acb None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:54.306970 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a0b8f4-228f-42b2-8038-ee7d2c420acb None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:54.307358 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a0b8f4-228f-42b2-8038-ee7d2c420acb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:54.339379 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-00a0b8f4-228f-42b2-8038-ee7d2c420acb None None] MFA Rules not processed for user `e58f8aa100724d448846f3c1722d66cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:54.341304 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-00a0b8f4-228f-42b2-8038-ee7d2c420acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:54.364618 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2c425c-f69a-453f-b484-381d578fd358 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:54.373770 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5d53da-a981-40c4-ad95-a391e9b6babc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:54.373770 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5d53da-a981-40c4-ad95-a391e9b6babc service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:54.373770 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5d53da-a981-40c4-ad95-a391e9b6babc service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:54.373770 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5d53da-a981-40c4-ad95-a391e9b6babc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:54.440318 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1018/2020] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:54 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:21:54.441402 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a0b8f4-228f-42b2-8038-ee7d2c420acb None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:54.443561 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-00a0b8f4-228f-42b2-8038-ee7d2c420acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:54.457771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5d53da-a981-40c4-ad95-a391e9b6babc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:54.459480 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5d53da-a981-40c4-ad95-a391e9b6babc service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:54.471840 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1003/2021] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:21:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:21:54.522580 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a0b8f4-228f-42b2-8038-ee7d2c420acb None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:54.530915 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d3f4ce-5c6e-4dc7-80a6-1068b8a87aad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:54.531515 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d3f4ce-5c6e-4dc7-80a6-1068b8a87aad service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:54.531745 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d3f4ce-5c6e-4dc7-80a6-1068b8a87aad service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:54.532030 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d3f4ce-5c6e-4dc7-80a6-1068b8a87aad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:54.533913 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-a7d3f4ce-5c6e-4dc7-80a6-1068b8a87aad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:54.619931 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d3f4ce-5c6e-4dc7-80a6-1068b8a87aad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:54.621490 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d3f4ce-5c6e-4dc7-80a6-1068b8a87aad service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:54.636747 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1019/2022] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:21:54 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:21:54.978798 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5d53da-a981-40c4-ad95-a391e9b6babc service neutron] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:54.981282 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-bb5d53da-a981-40c4-ad95-a391e9b6babc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:54.998485 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5d53da-a981-40c4-ad95-a391e9b6babc service neutron] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:55.003615 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f05c055-6401-4526-abbd-eda14f608eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:55.004517 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f05c055-6401-4526-abbd-eda14f608eba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:55.004827 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f05c055-6401-4526-abbd-eda14f608eba admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:55.005099 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f05c055-6401-4526-abbd-eda14f608eba admin admin] PATH_INFO: `/v3/projects/44c6b29b0b3542c488e7b9428c61ca4c` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:55.006392 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f05c055-6401-4526-abbd-eda14f608eba admin admin] RBAC: Authorizing `identity:delete_project(project_id=44c6b29b0b3542c488e7b9428c61ca4c)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:55.008597 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f05c055-6401-4526-abbd-eda14f608eba admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:55.037994 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:55.091909 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d3f4ce-5c6e-4dc7-80a6-1068b8a87aad service cinder] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:55.093334 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-a7d3f4ce-5c6e-4dc7-80a6-1068b8a87aad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:55.118102 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-6f05c055-6401-4526-abbd-eda14f608eba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44c6b29b0b3542c488e7b9428c61ca4c', 'request_id': 'req-6f05c055-6401-4526-abbd-eda14f608eba'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:55.118837 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d3f4ce-5c6e-4dc7-80a6-1068b8a87aad service cinder] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:55.152194 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:55.184066 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2254d6eb-42f4-4043-890c-af5c744b36fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:55.184985 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2254d6eb-42f4-4043-890c-af5c744b36fd service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:55.185350 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2254d6eb-42f4-4043-890c-af5c744b36fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:55.185725 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2254d6eb-42f4-4043-890c-af5c744b36fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:55.221889 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-6f05c055-6401-4526-abbd-eda14f608eba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44c6b29b0b3542c488e7b9428c61ca4c', 'request_id': 'req-6f05c055-6401-4526-abbd-eda14f608eba'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:55.229996 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-6f05c055-6401-4526-abbd-eda14f608eba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44c6b29b0b3542c488e7b9428c61ca4c', 'request_id': 'req-6f05c055-6401-4526-abbd-eda14f608eba'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:55.244639 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-6f05c055-6401-4526-abbd-eda14f608eba admin admin] The token cache is being invalidate because project 44c6b29b0b3542c488e7b9428c61ca4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:21:55.244966 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-6f05c055-6401-4526-abbd-eda14f608eba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:55.245789 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-6f05c055-6401-4526-abbd-eda14f608eba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:55.248433 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1004/2023] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:21:54 2022] DELETE /identity/v3/projects/44c6b29b0b3542c488e7b9428c61ca4c => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:55.252366 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f05c055-6401-4526-abbd-eda14f608eba admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:55.253694 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2254d6eb-42f4-4043-890c-af5c744b36fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:55.253816 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-6f05c055-6401-4526-abbd-eda14f608eba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:55.255471 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2254d6eb-42f4-4043-890c-af5c744b36fd service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:55.274886 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1020/2024] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:21:55 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 02 19:21:55.288235 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2254d6eb-42f4-4043-890c-af5c744b36fd service cinder] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:55.379970 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2254d6eb-42f4-4043-890c-af5c744b36fd service cinder] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:55.390607 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecd49ac-0c37-4e58-b8f1-e7aa04b0cf98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:55.400842 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecd49ac-0c37-4e58-b8f1-e7aa04b0cf98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:55.400842 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecd49ac-0c37-4e58-b8f1-e7aa04b0cf98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:55.400842 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecd49ac-0c37-4e58-b8f1-e7aa04b0cf98 admin admin] PATH_INFO: `/v3/users/e58f8aa100724d448846f3c1722d66cb` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:55.400842 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecd49ac-0c37-4e58-b8f1-e7aa04b0cf98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e58f8aa100724d448846f3c1722d66cb)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:55.400842 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecd49ac-0c37-4e58-b8f1-e7aa04b0cf98 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:55.406499 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f05c055-6401-4526-abbd-eda14f608eba admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:55.423212 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029fcca0-a413-45b1-a6d3-37ca769150ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:55.423212 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029fcca0-a413-45b1-a6d3-37ca769150ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:55.423212 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029fcca0-a413-45b1-a6d3-37ca769150ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:55.423212 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029fcca0-a413-45b1-a6d3-37ca769150ac admin admin] PATH_INFO: `/v3/users/7b56ba129e384934a10a73af29871413` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:55.423736 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029fcca0-a413-45b1-a6d3-37ca769150ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b56ba129e384934a10a73af29871413)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:55.423736 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029fcca0-a413-45b1-a6d3-37ca769150ac admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:55.550491 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-0ecd49ac-0c37-4e58-b8f1-e7aa04b0cf98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e58f8aa100724d448846f3c1722d66cb'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:55.557797 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-0ecd49ac-0c37-4e58-b8f1-e7aa04b0cf98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e58f8aa100724d448846f3c1722d66cb'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:55.601454 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-029fcca0-a413-45b1-a6d3-37ca769150ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b56ba129e384934a10a73af29871413'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:55.608684 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-029fcca0-a413-45b1-a6d3-37ca769150ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b56ba129e384934a10a73af29871413'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:55.609798 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-029fcca0-a413-45b1-a6d3-37ca769150ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b56ba129e384934a10a73af29871413'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:55.610396 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-0ecd49ac-0c37-4e58-b8f1-e7aa04b0cf98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e58f8aa100724d448846f3c1722d66cb'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:55.611094 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-0ecd49ac-0c37-4e58-b8f1-e7aa04b0cf98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e58f8aa100724d448846f3c1722d66cb'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:55.611733 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-0ecd49ac-0c37-4e58-b8f1-e7aa04b0cf98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e58f8aa100724d448846f3c1722d66cb'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:55.626017 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:55.631063 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1021/2025] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:21:55 2022] DELETE /identity/v3/users/e58f8aa100724d448846f3c1722d66cb => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:55.635627 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448c06bd-6c25-457c-93a7-b0dc901ff2ce None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:55.635942 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448c06bd-6c25-457c-93a7-b0dc901ff2ce None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:55.636238 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448c06bd-6c25-457c-93a7-b0dc901ff2ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:55.654290 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-448c06bd-6c25-457c-93a7-b0dc901ff2ce None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:55.655510 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-448c06bd-6c25-457c-93a7-b0dc901ff2ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:55.712639 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1022/2026] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 19:21:55.722521 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448c06bd-6c25-457c-93a7-b0dc901ff2ce None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:55.723890 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-448c06bd-6c25-457c-93a7-b0dc901ff2ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:55.775895 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-029fcca0-a413-45b1-a6d3-37ca769150ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b56ba129e384934a10a73af29871413'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:55.776547 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-029fcca0-a413-45b1-a6d3-37ca769150ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b56ba129e384934a10a73af29871413'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:55.787952 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448c06bd-6c25-457c-93a7-b0dc901ff2ce None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:55.791612 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55ccd91-444c-4717-a5df-a756487eaefb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:55.792268 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55ccd91-444c-4717-a5df-a756487eaefb service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:55.792547 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55ccd91-444c-4717-a5df-a756487eaefb service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:55.792824 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55ccd91-444c-4717-a5df-a756487eaefb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:55.796285 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1005/2027] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:21:55 2022] DELETE /identity/v3/users/7b56ba129e384934a10a73af29871413 => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:55.802789 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55ccd91-444c-4717-a5df-a756487eaefb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:55.805223 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55ccd91-444c-4717-a5df-a756487eaefb service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:55.814936 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1023/2028] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:21:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 02 19:21:56.287587 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029fcca0-a413-45b1-a6d3-37ca769150ac admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:56.289555 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-029fcca0-a413-45b1-a6d3-37ca769150ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:56.327991 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:56.337224 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55ccd91-444c-4717-a5df-a756487eaefb service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:56.337224 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-a55ccd91-444c-4717-a5df-a756487eaefb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:56.366350 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55ccd91-444c-4717-a5df-a756487eaefb service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:56.384584 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029fcca0-a413-45b1-a6d3-37ca769150ac admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:56.390105 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a28732-839a-4a2c-be8e-899d6dc0cbdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:56.391015 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a28732-839a-4a2c-be8e-899d6dc0cbdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:56.391323 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a28732-839a-4a2c-be8e-899d6dc0cbdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:56.391599 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a28732-839a-4a2c-be8e-899d6dc0cbdc admin admin] PATH_INFO: `/v3/projects/907e8b96738746b9b10db27a679c45d1` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:56.392932 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a28732-839a-4a2c-be8e-899d6dc0cbdc admin admin] RBAC: Authorizing `identity:delete_project(project_id=907e8b96738746b9b10db27a679c45d1)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:56.394959 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a28732-839a-4a2c-be8e-899d6dc0cbdc admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:56.443510 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec41bcab-33ff-4721-83ea-7524768738f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:56.443510 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec41bcab-33ff-4721-83ea-7524768738f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:56.443510 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec41bcab-33ff-4721-83ea-7524768738f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:56.443759 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec41bcab-33ff-4721-83ea-7524768738f0 admin admin] PATH_INFO: `/v3/projects/462d3a4a6a4249d980ff6bbb77ee5dee` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:56.445099 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec41bcab-33ff-4721-83ea-7524768738f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=462d3a4a6a4249d980ff6bbb77ee5dee)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:56.447217 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec41bcab-33ff-4721-83ea-7524768738f0 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:56.531711 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-79a28732-839a-4a2c-be8e-899d6dc0cbdc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '907e8b96738746b9b10db27a679c45d1', 'request_id': 'req-79a28732-839a-4a2c-be8e-899d6dc0cbdc'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:56.574810 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:56.584502 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-ec41bcab-33ff-4721-83ea-7524768738f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '462d3a4a6a4249d980ff6bbb77ee5dee', 'request_id': 'req-ec41bcab-33ff-4721-83ea-7524768738f0'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:56.616911 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-79a28732-839a-4a2c-be8e-899d6dc0cbdc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '907e8b96738746b9b10db27a679c45d1', 'request_id': 'req-79a28732-839a-4a2c-be8e-899d6dc0cbdc'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:56.623021 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-79a28732-839a-4a2c-be8e-899d6dc0cbdc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '907e8b96738746b9b10db27a679c45d1', 'request_id': 'req-79a28732-839a-4a2c-be8e-899d6dc0cbdc'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:56.635063 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-79a28732-839a-4a2c-be8e-899d6dc0cbdc admin admin] The token cache is being invalidate because project 907e8b96738746b9b10db27a679c45d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:21:56.635373 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-79a28732-839a-4a2c-be8e-899d6dc0cbdc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:56.636022 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-79a28732-839a-4a2c-be8e-899d6dc0cbdc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:56.637662 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1006/2029] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:21:56 2022] DELETE /identity/v3/projects/907e8b96738746b9b10db27a679c45d1 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:21:56.641351 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a28732-839a-4a2c-be8e-899d6dc0cbdc admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:56.642773 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-79a28732-839a-4a2c-be8e-899d6dc0cbdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:56.679291 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-ec41bcab-33ff-4721-83ea-7524768738f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '462d3a4a6a4249d980ff6bbb77ee5dee', 'request_id': 'req-ec41bcab-33ff-4721-83ea-7524768738f0'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:56.685759 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-ec41bcab-33ff-4721-83ea-7524768738f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '462d3a4a6a4249d980ff6bbb77ee5dee', 'request_id': 'req-ec41bcab-33ff-4721-83ea-7524768738f0'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:56.698188 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-ec41bcab-33ff-4721-83ea-7524768738f0 admin admin] The token cache is being invalidate because project 462d3a4a6a4249d980ff6bbb77ee5dee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:21:56.698481 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-ec41bcab-33ff-4721-83ea-7524768738f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:56.699164 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-ec41bcab-33ff-4721-83ea-7524768738f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:56.700841 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1024/2030] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:21:56 2022] DELETE /identity/v3/projects/462d3a4a6a4249d980ff6bbb77ee5dee => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 02 19:21:56.706971 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939e58e-e92c-4c38-806f-0ae9bc300df6 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:56.707261 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939e58e-e92c-4c38-806f-0ae9bc300df6 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:56.707539 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939e58e-e92c-4c38-806f-0ae9bc300df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:56.715664 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a28732-839a-4a2c-be8e-899d6dc0cbdc admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:56.725594 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-6939e58e-e92c-4c38-806f-0ae9bc300df6 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:56.726821 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-6939e58e-e92c-4c38-806f-0ae9bc300df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:56.773550 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712a0ec3-72aa-4e54-a3cd-f0117b9b9444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:56.774445 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712a0ec3-72aa-4e54-a3cd-f0117b9b9444 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:56.774735 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712a0ec3-72aa-4e54-a3cd-f0117b9b9444 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:56.775097 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712a0ec3-72aa-4e54-a3cd-f0117b9b9444 admin admin] PATH_INFO: `/v3/users/591ec8887c0c4bc1aefcbe6ae4809b13` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:56.776324 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712a0ec3-72aa-4e54-a3cd-f0117b9b9444 admin admin] RBAC: Authorizing `identity:delete_user(user_id=591ec8887c0c4bc1aefcbe6ae4809b13)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:56.778491 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712a0ec3-72aa-4e54-a3cd-f0117b9b9444 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:56.783956 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:56.792147 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1025/2031] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 19:21:56.796207 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6461a051-6b65-435b-a763-cf65944da426 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:56.796501 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6461a051-6b65-435b-a763-cf65944da426 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:56.796787 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6461a051-6b65-435b-a763-cf65944da426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:56.815045 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-6461a051-6b65-435b-a763-cf65944da426 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:56.816257 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-6461a051-6b65-435b-a763-cf65944da426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:56.824991 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1026/2032] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 19:21:56.828957 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6461a051-6b65-435b-a763-cf65944da426 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:56.837925 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6461a051-6b65-435b-a763-cf65944da426 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:56.842644 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa61bd0-8985-4089-b036-8963cfce4162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:56.843327 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa61bd0-8985-4089-b036-8963cfce4162 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:56.843617 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa61bd0-8985-4089-b036-8963cfce4162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:56.843896 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa61bd0-8985-4089-b036-8963cfce4162 admin admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:56.844631 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa61bd0-8985-4089-b036-8963cfce4162 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:21:56.844942 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa61bd0-8985-4089-b036-8963cfce4162 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:56.846562 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa61bd0-8985-4089-b036-8963cfce4162 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:56.858482 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1027/2033] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 2 19:21:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:21:56.862775 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa61bd0-8985-4089-b036-8963cfce4162 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:56.873343 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa61bd0-8985-4089-b036-8963cfce4162 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:56.878349 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838a9178-0bf4-47d3-99fb-5a9253484d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:56.879040 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838a9178-0bf4-47d3-99fb-5a9253484d04 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:56.879329 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838a9178-0bf4-47d3-99fb-5a9253484d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:56.879612 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838a9178-0bf4-47d3-99fb-5a9253484d04 admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:56.880336 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838a9178-0bf4-47d3-99fb-5a9253484d04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:56.882947 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838a9178-0bf4-47d3-99fb-5a9253484d04 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:56.932625 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-712a0ec3-72aa-4e54-a3cd-f0117b9b9444 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '591ec8887c0c4bc1aefcbe6ae4809b13'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:56.939591 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-712a0ec3-72aa-4e54-a3cd-f0117b9b9444 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '591ec8887c0c4bc1aefcbe6ae4809b13'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:56.940306 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-712a0ec3-72aa-4e54-a3cd-f0117b9b9444 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '591ec8887c0c4bc1aefcbe6ae4809b13'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:57.012097 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1028/2034] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:21:56 2022] POST /identity/v3/projects => generated 389 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 02 19:21:57.016382 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838a9178-0bf4-47d3-99fb-5a9253484d04 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:57.018054 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-838a9178-0bf4-47d3-99fb-5a9253484d04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:57.027185 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-712a0ec3-72aa-4e54-a3cd-f0117b9b9444 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '591ec8887c0c4bc1aefcbe6ae4809b13'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:57.027838 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-712a0ec3-72aa-4e54-a3cd-f0117b9b9444 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '591ec8887c0c4bc1aefcbe6ae4809b13'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:57.046945 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1007/2035] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:21:56 2022] DELETE /identity/v3/users/591ec8887c0c4bc1aefcbe6ae4809b13 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 02 19:21:57.086541 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838a9178-0bf4-47d3-99fb-5a9253484d04 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:57.131095 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:57.149471 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1871d8-189e-41f2-aec0-3724dcd421cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:57.150185 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1871d8-189e-41f2-aec0-3724dcd421cc admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:57.150476 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1871d8-189e-41f2-aec0-3724dcd421cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:57.150821 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1871d8-189e-41f2-aec0-3724dcd421cc admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:57.151521 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1871d8-189e-41f2-aec0-3724dcd421cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:57.153518 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1871d8-189e-41f2-aec0-3724dcd421cc admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:57.233488 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1029/2036] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:21:57 2022] POST /identity/v3/users => generated 295 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 02 19:21:57.238017 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712a0ec3-72aa-4e54-a3cd-f0117b9b9444 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:57.258651 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712a0ec3-72aa-4e54-a3cd-f0117b9b9444 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:57.259475 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f831ab56-5360-4071-a243-8571d0d6ba38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:57.260330 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831ab56-5360-4071-a243-8571d0d6ba38 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:57.260679 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831ab56-5360-4071-a243-8571d0d6ba38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:57.261027 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831ab56-5360-4071-a243-8571d0d6ba38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:57.261831 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f831ab56-5360-4071-a243-8571d0d6ba38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:57.263563 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f831ab56-5360-4071-a243-8571d0d6ba38 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:57.275852 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1008/2037] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:21:57 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:21:57.279635 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1871d8-189e-41f2-aec0-3724dcd421cc admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:57.291429 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1871d8-189e-41f2-aec0-3724dcd421cc admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:57.296552 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63b98e7-57f5-4646-af5c-a6499624f011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:57.298570 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63b98e7-57f5-4646-af5c-a6499624f011 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:57.298570 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63b98e7-57f5-4646-af5c-a6499624f011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:57.298570 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63b98e7-57f5-4646-af5c-a6499624f011 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:57.298861 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63b98e7-57f5-4646-af5c-a6499624f011 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:57.304683 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63b98e7-57f5-4646-af5c-a6499624f011 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:57.320712 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1030/2038] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:21:57 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:21:57.321702 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f831ab56-5360-4071-a243-8571d0d6ba38 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:57.332868 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f831ab56-5360-4071-a243-8571d0d6ba38 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:57.338802 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e486a-b7ab-4ddf-a815-49a608d3bb6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:57.339773 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e486a-b7ab-4ddf-a815-49a608d3bb6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:57.340159 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e486a-b7ab-4ddf-a815-49a608d3bb6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:57.340739 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e486a-b7ab-4ddf-a815-49a608d3bb6f admin admin] PATH_INFO: `/v3/projects/21706943ae514136899b35cec41bd931/users/c587274f4a0945abaf3a4ac48f3fed29/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:57.359130 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9e486a-b7ab-4ddf-a815-49a608d3bb6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=21706943ae514136899b35cec41bd931, user_id=c587274f4a0945abaf3a4ac48f3fed29, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:57.361890 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9e486a-b7ab-4ddf-a815-49a608d3bb6f admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:57.439574 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1009/2039] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:21:57 2022] PUT /identity/v3/projects/21706943ae514136899b35cec41bd931/users/c587274f4a0945abaf3a4ac48f3fed29/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 02 19:21:57.451318 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176f226-7327-4378-ae17-79dcade46d68 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:57.451616 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176f226-7327-4378-ae17-79dcade46d68 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:57.451903 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176f226-7327-4378-ae17-79dcade46d68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:57.470800 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e486a-b7ab-4ddf-a815-49a608d3bb6f admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:57.472907 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-de9e486a-b7ab-4ddf-a815-49a608d3bb6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:57.493536 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-7176f226-7327-4378-ae17-79dcade46d68 None None] MFA Rules not processed for user `c587274f4a0945abaf3a4ac48f3fed29`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:57.495143 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-7176f226-7327-4378-ae17-79dcade46d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:57.557496 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e486a-b7ab-4ddf-a815-49a608d3bb6f admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:57.562289 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43912a50-cbf0-4e0d-bc51-d89c344d6bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:57.563241 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43912a50-cbf0-4e0d-bc51-d89c344d6bd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:57.563528 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43912a50-cbf0-4e0d-bc51-d89c344d6bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:57.563802 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43912a50-cbf0-4e0d-bc51-d89c344d6bd0 admin admin] PATH_INFO: `/v3/projects/6c6cc5f20c4e4df4970550bea9f27282` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:57.565206 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43912a50-cbf0-4e0d-bc51-d89c344d6bd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c6cc5f20c4e4df4970550bea9f27282)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:57.566145 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1031/2040] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:21:57.567158 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43912a50-cbf0-4e0d-bc51-d89c344d6bd0 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:57.576839 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7176f226-7327-4378-ae17-79dcade46d68 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:57.578222 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-7176f226-7327-4378-ae17-79dcade46d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:57.637781 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7176f226-7327-4378-ae17-79dcade46d68 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:57.641538 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09587928-d4d8-40d5-a7e3-92cee2e9d2e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:57.642192 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09587928-d4d8-40d5-a7e3-92cee2e9d2e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:57.642475 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09587928-d4d8-40d5-a7e3-92cee2e9d2e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:57.642780 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09587928-d4d8-40d5-a7e3-92cee2e9d2e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:57.652423 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09587928-d4d8-40d5-a7e3-92cee2e9d2e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:57.653920 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09587928-d4d8-40d5-a7e3-92cee2e9d2e9 service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:57.665742 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1032/2041] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:21:57 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 02 19:21:57.679349 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-43912a50-cbf0-4e0d-bc51-d89c344d6bd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c6cc5f20c4e4df4970550bea9f27282', 'request_id': 'req-43912a50-cbf0-4e0d-bc51-d89c344d6bd0'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:57.682962 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:57.775995 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-43912a50-cbf0-4e0d-bc51-d89c344d6bd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c6cc5f20c4e4df4970550bea9f27282', 'request_id': 'req-43912a50-cbf0-4e0d-bc51-d89c344d6bd0'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:57.783039 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-43912a50-cbf0-4e0d-bc51-d89c344d6bd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c6cc5f20c4e4df4970550bea9f27282', 'request_id': 'req-43912a50-cbf0-4e0d-bc51-d89c344d6bd0'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:57.794991 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-43912a50-cbf0-4e0d-bc51-d89c344d6bd0 admin admin] The token cache is being invalidate because project 6c6cc5f20c4e4df4970550bea9f27282 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:21:57.795288 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-43912a50-cbf0-4e0d-bc51-d89c344d6bd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:57.795984 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-43912a50-cbf0-4e0d-bc51-d89c344d6bd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:21:57.801697 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1010/2042] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:21:57 2022] DELETE /identity/v3/projects/6c6cc5f20c4e4df4970550bea9f27282 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 02 19:21:57.804372 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816fffd9-c611-4c9a-b78a-a403bf8dc871 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:57.804857 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816fffd9-c611-4c9a-b78a-a403bf8dc871 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:57.805040 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816fffd9-c611-4c9a-b78a-a403bf8dc871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:57.825833 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-816fffd9-c611-4c9a-b78a-a403bf8dc871 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:57.827090 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-816fffd9-c611-4c9a-b78a-a403bf8dc871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:57.877189 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:21:57.893610 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1033/2043] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:21:57.902645 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0eb382-0f07-4bb4-b180-ab3d1fd667cc None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:57.902966 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0eb382-0f07-4bb4-b180-ab3d1fd667cc None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:57.903260 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0eb382-0f07-4bb4-b180-ab3d1fd667cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:57.923443 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-8d0eb382-0f07-4bb4-b180-ab3d1fd667cc None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:57.925914 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-8d0eb382-0f07-4bb4-b180-ab3d1fd667cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:57.934008 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1011/2044] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:21:57.938475 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816fffd9-c611-4c9a-b78a-a403bf8dc871 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:57.950719 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816fffd9-c611-4c9a-b78a-a403bf8dc871 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:57.956738 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e0d98-cb70-4c1a-9374-1b82ce3b2790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:57.957449 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e0d98-cb70-4c1a-9374-1b82ce3b2790 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:57.957739 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e0d98-cb70-4c1a-9374-1b82ce3b2790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:57.958012 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e0d98-cb70-4c1a-9374-1b82ce3b2790 admin admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:57.959459 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9e0d98-cb70-4c1a-9374-1b82ce3b2790 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:21:57.959767 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9e0d98-cb70-4c1a-9374-1b82ce3b2790 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:57.961505 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9e0d98-cb70-4c1a-9374-1b82ce3b2790 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:57.978525 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1034/2045] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 2 19:21:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:21:57.987929 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0eb382-0f07-4bb4-b180-ab3d1fd667cc None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:57.999334 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0eb382-0f07-4bb4-b180-ab3d1fd667cc None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:58.004766 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67056e3e-c922-409d-bc4a-c8afa717595e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:58.006035 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67056e3e-c922-409d-bc4a-c8afa717595e admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:58.006411 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67056e3e-c922-409d-bc4a-c8afa717595e admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:58.006561 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67056e3e-c922-409d-bc4a-c8afa717595e admin admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:58.007331 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67056e3e-c922-409d-bc4a-c8afa717595e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:58.009323 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67056e3e-c922-409d-bc4a-c8afa717595e admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:58.136826 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1012/2046] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:21:57 2022] POST /identity/v3/projects => generated 373 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:21:58.138597 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e0d98-cb70-4c1a-9374-1b82ce3b2790 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:58.149954 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e0d98-cb70-4c1a-9374-1b82ce3b2790 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:58.220393 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99acb49-2eec-452d-ac9b-2bda21184691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:58.221096 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99acb49-2eec-452d-ac9b-2bda21184691 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:58.221386 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99acb49-2eec-452d-ac9b-2bda21184691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:58.221669 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99acb49-2eec-452d-ac9b-2bda21184691 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:58.222400 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99acb49-2eec-452d-ac9b-2bda21184691 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:58.224537 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99acb49-2eec-452d-ac9b-2bda21184691 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:58.321320 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1035/2047] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:21:58 2022] POST /identity/v3/users => generated 287 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:21:58.325843 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67056e3e-c922-409d-bc4a-c8afa717595e admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:58.337328 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67056e3e-c922-409d-bc4a-c8afa717595e admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:58.342690 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6b0339-3626-49f8-beb6-7dfe062c918d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:58.343640 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6b0339-3626-49f8-beb6-7dfe062c918d admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:58.344203 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6b0339-3626-49f8-beb6-7dfe062c918d admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:58.344594 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6b0339-3626-49f8-beb6-7dfe062c918d admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:58.345686 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6b0339-3626-49f8-beb6-7dfe062c918d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:58.347640 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6b0339-3626-49f8-beb6-7dfe062c918d admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:58.368109 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1013/2048] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:21:58 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:21:58.372562 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99acb49-2eec-452d-ac9b-2bda21184691 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:58.385557 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99acb49-2eec-452d-ac9b-2bda21184691 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:58.391061 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5ce74e-fe0c-4a5c-99bf-a797093ac812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:58.391700 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ce74e-fe0c-4a5c-99bf-a797093ac812 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:58.391994 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ce74e-fe0c-4a5c-99bf-a797093ac812 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:58.392249 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ce74e-fe0c-4a5c-99bf-a797093ac812 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:58.394663 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5ce74e-fe0c-4a5c-99bf-a797093ac812 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:58.396501 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5ce74e-fe0c-4a5c-99bf-a797093ac812 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:58.411364 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1036/2049] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:21:58 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:21:58.417271 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6b0339-3626-49f8-beb6-7dfe062c918d admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:58.443760 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6b0339-3626-49f8-beb6-7dfe062c918d admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:58.449968 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a747afc8-1856-497a-874b-d8ca416b999d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:58.451000 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a747afc8-1856-497a-874b-d8ca416b999d admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:58.451286 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a747afc8-1856-497a-874b-d8ca416b999d admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:58.451563 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a747afc8-1856-497a-874b-d8ca416b999d admin admin] PATH_INFO: `/v3/projects/1e2cdcd22e66456da3a49a5333f64de2/users/d38ac88237f54a0895a8c04e2a9290a9/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:58.471254 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a747afc8-1856-497a-874b-d8ca416b999d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e2cdcd22e66456da3a49a5333f64de2, user_id=d38ac88237f54a0895a8c04e2a9290a9, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:58.475468 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a747afc8-1856-497a-874b-d8ca416b999d admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:58.564875 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1014/2050] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:21:58 2022] PUT /identity/v3/projects/1e2cdcd22e66456da3a49a5333f64de2/users/d38ac88237f54a0895a8c04e2a9290a9/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 02 19:21:58.574638 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8373f8-39fd-4086-b833-7cb1438fa93f None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:58.574956 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8373f8-39fd-4086-b833-7cb1438fa93f None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:58.575244 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8373f8-39fd-4086-b833-7cb1438fa93f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:58.601837 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-7b8373f8-39fd-4086-b833-7cb1438fa93f None None] MFA Rules not processed for user `d38ac88237f54a0895a8c04e2a9290a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:21:58.603081 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-7b8373f8-39fd-4086-b833-7cb1438fa93f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:58.666396 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1037/2051] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:21:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:21:58.676791 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a747afc8-1856-497a-874b-d8ca416b999d admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:21:58.679138 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-a747afc8-1856-497a-874b-d8ca416b999d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:21:58.734815 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a747afc8-1856-497a-874b-d8ca416b999d admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:21:58.738426 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efda6e8d-dbf6-4292-bb4d-17e7fcdf790f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:21:58.739117 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda6e8d-dbf6-4292-bb4d-17e7fcdf790f service cinder] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:21:58.739395 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda6e8d-dbf6-4292-bb4d-17e7fcdf790f service cinder] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:21:58.739668 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda6e8d-dbf6-4292-bb4d-17e7fcdf790f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:21:58.749002 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efda6e8d-dbf6-4292-bb4d-17e7fcdf790f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:21:58.750420 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efda6e8d-dbf6-4292-bb4d-17e7fcdf790f service cinder] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:21:58.759412 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1015/2052] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:21:58 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:22:00.549570 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3003315-1461-4b31-8334-78493777d841 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:00.550173 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3003315-1461-4b31-8334-78493777d841 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:00.551981 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3003315-1461-4b31-8334-78493777d841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:00.577891 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-f3003315-1461-4b31-8334-78493777d841 None None] MFA Rules not processed for user `d38ac88237f54a0895a8c04e2a9290a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:22:00.579016 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-f3003315-1461-4b31-8334-78493777d841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:00.591242 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1038/2053] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:22:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:22:01.063557 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efda6e8d-dbf6-4292-bb4d-17e7fcdf790f service cinder] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:01.065058 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-efda6e8d-dbf6-4292-bb4d-17e7fcdf790f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:01.078404 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efda6e8d-dbf6-4292-bb4d-17e7fcdf790f service cinder] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:01.094499 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a002c2-7d61-41b1-a7ee-2b52f1d714fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:01.095724 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a002c2-7d61-41b1-a7ee-2b52f1d714fb service cinder] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:01.096312 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a002c2-7d61-41b1-a7ee-2b52f1d714fb service cinder] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:01.096737 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a002c2-7d61-41b1-a7ee-2b52f1d714fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:01.102554 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-73a002c2-7d61-41b1-a7ee-2b52f1d714fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:01.241840 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a002c2-7d61-41b1-a7ee-2b52f1d714fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:01.245276 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a002c2-7d61-41b1-a7ee-2b52f1d714fb service cinder] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:01.263332 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1016/2054] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:22:01 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 02 19:22:03.008013 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3003315-1461-4b31-8334-78493777d841 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:03.017540 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3003315-1461-4b31-8334-78493777d841 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:03.021687 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ac22d8-4da2-416b-a597-eaedd8af1b8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:03.022341 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ac22d8-4da2-416b-a597-eaedd8af1b8f service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:03.022623 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ac22d8-4da2-416b-a597-eaedd8af1b8f service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:03.022934 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ac22d8-4da2-416b-a597-eaedd8af1b8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:03.024782 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-63ac22d8-4da2-416b-a597-eaedd8af1b8f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:03.063685 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:22:03.088619 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ac22d8-4da2-416b-a597-eaedd8af1b8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:03.090063 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ac22d8-4da2-416b-a597-eaedd8af1b8f service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:03.099688 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1039/2055] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:22:03 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:22:11.254819 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:DELETE=3,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:22:13.130868 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=12 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:22:13.988392 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a002c2-7d61-41b1-a7ee-2b52f1d714fb service cinder] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:13.994957 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-73a002c2-7d61-41b1-a7ee-2b52f1d714fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:14.056829 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a002c2-7d61-41b1-a7ee-2b52f1d714fb service cinder] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:14.061385 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e8825e-8c9d-47be-a028-534653f06678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:14.062269 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8825e-8c9d-47be-a028-534653f06678 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:14.062558 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8825e-8c9d-47be-a028-534653f06678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:14.062873 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8825e-8c9d-47be-a028-534653f06678 admin admin] PATH_INFO: `/v3/users/c68940b1a426492eb94d0a9fc0ea0c3b` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:14.064112 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e8825e-8c9d-47be-a028-534653f06678 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c68940b1a426492eb94d0a9fc0ea0c3b)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:14.066037 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e8825e-8c9d-47be-a028-534653f06678 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:14.237681 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-17e8825e-8c9d-47be-a028-534653f06678 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c68940b1a426492eb94d0a9fc0ea0c3b'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:14.246273 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-17e8825e-8c9d-47be-a028-534653f06678 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c68940b1a426492eb94d0a9fc0ea0c3b'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:14.247110 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-17e8825e-8c9d-47be-a028-534653f06678 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c68940b1a426492eb94d0a9fc0ea0c3b'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:14.301714 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-17e8825e-8c9d-47be-a028-534653f06678 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c68940b1a426492eb94d0a9fc0ea0c3b'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:14.302342 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-17e8825e-8c9d-47be-a028-534653f06678 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c68940b1a426492eb94d0a9fc0ea0c3b'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:14.320996 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1017/2056] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:22:13 2022] DELETE /identity/v3/users/c68940b1a426492eb94d0a9fc0ea0c3b => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:22:14.544002 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ac22d8-4da2-416b-a597-eaedd8af1b8f service cinder] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:14.556920 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ac22d8-4da2-416b-a597-eaedd8af1b8f service cinder] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:14.612706 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35aac7e-bf8f-49d7-aac6-8d42bfd94aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:14.613567 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35aac7e-bf8f-49d7-aac6-8d42bfd94aed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:14.613852 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35aac7e-bf8f-49d7-aac6-8d42bfd94aed admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:14.614152 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35aac7e-bf8f-49d7-aac6-8d42bfd94aed admin admin] PATH_INFO: `/v3/projects/738f5dc47a184fd2962b9895a3827dd3` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:14.615371 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35aac7e-bf8f-49d7-aac6-8d42bfd94aed admin admin] RBAC: Authorizing `identity:delete_project(project_id=738f5dc47a184fd2962b9895a3827dd3)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:14.617566 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35aac7e-bf8f-49d7-aac6-8d42bfd94aed admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:14.731502 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-b35aac7e-bf8f-49d7-aac6-8d42bfd94aed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '738f5dc47a184fd2962b9895a3827dd3', 'request_id': 'req-b35aac7e-bf8f-49d7-aac6-8d42bfd94aed'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:14.798709 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-b35aac7e-bf8f-49d7-aac6-8d42bfd94aed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '738f5dc47a184fd2962b9895a3827dd3', 'request_id': 'req-b35aac7e-bf8f-49d7-aac6-8d42bfd94aed'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:14.805761 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-b35aac7e-bf8f-49d7-aac6-8d42bfd94aed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '738f5dc47a184fd2962b9895a3827dd3', 'request_id': 'req-b35aac7e-bf8f-49d7-aac6-8d42bfd94aed'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:14.817832 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-b35aac7e-bf8f-49d7-aac6-8d42bfd94aed admin admin] The token cache is being invalidate because project 738f5dc47a184fd2962b9895a3827dd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:22:14.818884 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-b35aac7e-bf8f-49d7-aac6-8d42bfd94aed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:14.819610 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-b35aac7e-bf8f-49d7-aac6-8d42bfd94aed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:14.821366 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1040/2057] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:22:14 2022] DELETE /identity/v3/projects/738f5dc47a184fd2962b9895a3827dd3 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:22:14.826499 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72a07da-3663-42d2-a60a-df338dbcd7c9 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:14.832139 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72a07da-3663-42d2-a60a-df338dbcd7c9 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:14.832139 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72a07da-3663-42d2-a60a-df338dbcd7c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:14.848119 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-e72a07da-3663-42d2-a60a-df338dbcd7c9 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:22:14.849420 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-e72a07da-3663-42d2-a60a-df338dbcd7c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:14.911043 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1018/2058] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:22:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:22:14.919838 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fdf65a-bd92-4307-8fde-24bb395a56e3 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:14.920134 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fdf65a-bd92-4307-8fde-24bb395a56e3 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:14.920497 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fdf65a-bd92-4307-8fde-24bb395a56e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:14.946238 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-29fdf65a-bd92-4307-8fde-24bb395a56e3 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:22:14.947570 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-29fdf65a-bd92-4307-8fde-24bb395a56e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:14.955739 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1041/2059] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:22:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:22:14.960795 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72a07da-3663-42d2-a60a-df338dbcd7c9 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:14.972645 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72a07da-3663-42d2-a60a-df338dbcd7c9 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:14.977913 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d48b0f2-fc97-4725-b690-961c1698a077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:14.978581 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d48b0f2-fc97-4725-b690-961c1698a077 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:14.978908 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d48b0f2-fc97-4725-b690-961c1698a077 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:14.979194 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d48b0f2-fc97-4725-b690-961c1698a077 admin admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:14.979927 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d48b0f2-fc97-4725-b690-961c1698a077 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:22:14.980235 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d48b0f2-fc97-4725-b690-961c1698a077 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:14.981845 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d48b0f2-fc97-4725-b690-961c1698a077 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:14.994807 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1019/2060] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 2 19:22:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:22:14.999228 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fdf65a-bd92-4307-8fde-24bb395a56e3 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:15.009581 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fdf65a-bd92-4307-8fde-24bb395a56e3 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:15.014322 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f111f8fb-631c-4603-bf39-d202b55b6ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:15.015019 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f111f8fb-631c-4603-bf39-d202b55b6ee2 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:15.015304 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f111f8fb-631c-4603-bf39-d202b55b6ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:15.015580 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f111f8fb-631c-4603-bf39-d202b55b6ee2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:15.016312 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f111f8fb-631c-4603-bf39-d202b55b6ee2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:15.018214 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f111f8fb-631c-4603-bf39-d202b55b6ee2 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:15.093049 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1042/2061] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:22:14 2022] POST /identity/v3/projects => generated 389 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:22:15.096452 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d48b0f2-fc97-4725-b690-961c1698a077 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:15.107385 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d48b0f2-fc97-4725-b690-961c1698a077 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:15.111647 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:22:15.170950 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3644fd62-1526-4aae-85c3-17b6c68807b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:15.171877 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3644fd62-1526-4aae-85c3-17b6c68807b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:15.172251 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3644fd62-1526-4aae-85c3-17b6c68807b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:15.173332 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3644fd62-1526-4aae-85c3-17b6c68807b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:15.174231 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3644fd62-1526-4aae-85c3-17b6c68807b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:15.176187 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3644fd62-1526-4aae-85c3-17b6c68807b0 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:15.251303 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1020/2062] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:22:15 2022] POST /identity/v3/users => generated 295 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:22:15.256010 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f111f8fb-631c-4603-bf39-d202b55b6ee2 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:15.272443 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f111f8fb-631c-4603-bf39-d202b55b6ee2 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:15.277289 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2823d333-62ff-4d7e-ad56-d3aaab4f01f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:15.279023 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2823d333-62ff-4d7e-ad56-d3aaab4f01f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:15.279023 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2823d333-62ff-4d7e-ad56-d3aaab4f01f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:15.279023 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2823d333-62ff-4d7e-ad56-d3aaab4f01f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:15.279353 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2823d333-62ff-4d7e-ad56-d3aaab4f01f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:15.280893 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2823d333-62ff-4d7e-ad56-d3aaab4f01f6 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:15.295613 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1043/2063] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:22:15 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:22:15.299540 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3644fd62-1526-4aae-85c3-17b6c68807b0 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:15.307516 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3644fd62-1526-4aae-85c3-17b6c68807b0 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:15.312478 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbf2d9a-893c-44c1-b736-7ec99d1b7bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:15.313228 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf2d9a-893c-44c1-b736-7ec99d1b7bab admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:15.313454 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf2d9a-893c-44c1-b736-7ec99d1b7bab admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:15.313735 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf2d9a-893c-44c1-b736-7ec99d1b7bab admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:15.314473 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbf2d9a-893c-44c1-b736-7ec99d1b7bab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:15.316159 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbf2d9a-893c-44c1-b736-7ec99d1b7bab admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:15.330420 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1021/2064] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:22:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:22:15.335283 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2823d333-62ff-4d7e-ad56-d3aaab4f01f6 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:15.351119 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2823d333-62ff-4d7e-ad56-d3aaab4f01f6 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:15.354691 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0ed48d-4f76-4ec7-be35-6fdb76e84d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:15.355618 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0ed48d-4f76-4ec7-be35-6fdb76e84d8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:15.356067 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0ed48d-4f76-4ec7-be35-6fdb76e84d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:15.356339 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0ed48d-4f76-4ec7-be35-6fdb76e84d8b admin admin] PATH_INFO: `/v3/projects/a661f1650b0a48dca89696dc4846d28c/users/85a9b91215d34c68827144968041e700/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:15.373936 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0ed48d-4f76-4ec7-be35-6fdb76e84d8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a661f1650b0a48dca89696dc4846d28c, user_id=85a9b91215d34c68827144968041e700, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:15.376735 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0ed48d-4f76-4ec7-be35-6fdb76e84d8b admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:15.475789 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1044/2065] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:22:15 2022] PUT /identity/v3/projects/a661f1650b0a48dca89696dc4846d28c/users/85a9b91215d34c68827144968041e700/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:22:15.480212 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72282eb5-d8c1-4850-9e00-67194980f75f None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:15.480502 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72282eb5-d8c1-4850-9e00-67194980f75f None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:15.480777 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72282eb5-d8c1-4850-9e00-67194980f75f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:15.501759 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-72282eb5-d8c1-4850-9e00-67194980f75f None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:22:15.503602 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-72282eb5-d8c1-4850-9e00-67194980f75f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:15.571598 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1022/2066] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:22:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:22:15.584665 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0ed48d-4f76-4ec7-be35-6fdb76e84d8b admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:15.587546 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-8c0ed48d-4f76-4ec7-be35-6fdb76e84d8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:15.629754 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:22:15.657710 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0ed48d-4f76-4ec7-be35-6fdb76e84d8b admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:15.661579 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2343f7-79b0-4fa7-b588-d39f19939f27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:15.662931 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2343f7-79b0-4fa7-b588-d39f19939f27 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:15.663216 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2343f7-79b0-4fa7-b588-d39f19939f27 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:15.663495 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2343f7-79b0-4fa7-b588-d39f19939f27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:15.673734 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2343f7-79b0-4fa7-b588-d39f19939f27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:15.675212 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2343f7-79b0-4fa7-b588-d39f19939f27 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:15.684725 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1045/2067] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:22:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:22:17.720817 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72282eb5-d8c1-4850-9e00-67194980f75f None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:17.722345 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-72282eb5-d8c1-4850-9e00-67194980f75f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:17.761630 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72282eb5-d8c1-4850-9e00-67194980f75f None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:17.784283 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74d6afa-21f5-4547-8494-9a515bc1244b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:17.785392 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d6afa-21f5-4547-8494-9a515bc1244b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:17.786395 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d6afa-21f5-4547-8494-9a515bc1244b admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:17.787968 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d6afa-21f5-4547-8494-9a515bc1244b admin admin] PATH_INFO: `/v3/users/6e71a60cc0d8472ebc61d12219f55055` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:17.789646 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74d6afa-21f5-4547-8494-9a515bc1244b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e71a60cc0d8472ebc61d12219f55055)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:17.793284 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74d6afa-21f5-4547-8494-9a515bc1244b admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:18.154407 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-b74d6afa-21f5-4547-8494-9a515bc1244b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e71a60cc0d8472ebc61d12219f55055'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:18.171608 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-b74d6afa-21f5-4547-8494-9a515bc1244b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e71a60cc0d8472ebc61d12219f55055'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:18.174632 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-b74d6afa-21f5-4547-8494-9a515bc1244b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e71a60cc0d8472ebc61d12219f55055'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:18.304527 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-b74d6afa-21f5-4547-8494-9a515bc1244b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e71a60cc0d8472ebc61d12219f55055'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:18.305996 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-b74d6afa-21f5-4547-8494-9a515bc1244b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e71a60cc0d8472ebc61d12219f55055'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:18.334350 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:22:18.348947 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1023/2068] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:22:17 2022] DELETE /identity/v3/users/6e71a60cc0d8472ebc61d12219f55055 => generated 0 bytes in 629 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:22:18.353874 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71290fc4-a13a-4c8c-81ac-1de62d0756b9 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:18.354956 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71290fc4-a13a-4c8c-81ac-1de62d0756b9 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:18.355423 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71290fc4-a13a-4c8c-81ac-1de62d0756b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:18.383328 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-71290fc4-a13a-4c8c-81ac-1de62d0756b9 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:22:18.391506 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-71290fc4-a13a-4c8c-81ac-1de62d0756b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:18.509283 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1046/2069] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:22:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:22:18.559343 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74d6afa-21f5-4547-8494-9a515bc1244b admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:18.560987 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-b74d6afa-21f5-4547-8494-9a515bc1244b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:18.644862 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74d6afa-21f5-4547-8494-9a515bc1244b admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:18.649543 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba1eed-845e-4102-8062-3ee83ddbe71b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:18.656045 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba1eed-845e-4102-8062-3ee83ddbe71b service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:18.656594 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba1eed-845e-4102-8062-3ee83ddbe71b service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:18.657037 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba1eed-845e-4102-8062-3ee83ddbe71b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:18.673531 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ba1eed-845e-4102-8062-3ee83ddbe71b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:18.676552 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ba1eed-845e-4102-8062-3ee83ddbe71b service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:18.691830 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1024/2070] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:22:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:22:18.937295 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71290fc4-a13a-4c8c-81ac-1de62d0756b9 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:18.938409 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-71290fc4-a13a-4c8c-81ac-1de62d0756b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:18.954053 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71290fc4-a13a-4c8c-81ac-1de62d0756b9 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:18.959959 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876f9086-b68c-4a62-9553-b9864a49cc99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:18.960992 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876f9086-b68c-4a62-9553-b9864a49cc99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:18.961357 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876f9086-b68c-4a62-9553-b9864a49cc99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:18.961644 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876f9086-b68c-4a62-9553-b9864a49cc99 admin admin] PATH_INFO: `/v3/users/ea59c4e399f14500b5d85178acc8a075` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:18.963210 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876f9086-b68c-4a62-9553-b9864a49cc99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea59c4e399f14500b5d85178acc8a075)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:18.965340 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876f9086-b68c-4a62-9553-b9864a49cc99 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:19.185693 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-876f9086-b68c-4a62-9553-b9864a49cc99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea59c4e399f14500b5d85178acc8a075'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:19.194385 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-876f9086-b68c-4a62-9553-b9864a49cc99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea59c4e399f14500b5d85178acc8a075'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:19.290853 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-876f9086-b68c-4a62-9553-b9864a49cc99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea59c4e399f14500b5d85178acc8a075'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:19.291498 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-876f9086-b68c-4a62-9553-b9864a49cc99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea59c4e399f14500b5d85178acc8a075'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:19.292164 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-876f9086-b68c-4a62-9553-b9864a49cc99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea59c4e399f14500b5d85178acc8a075'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:19.333552 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1047/2071] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:22:18 2022] DELETE /identity/v3/users/ea59c4e399f14500b5d85178acc8a075 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:22:19.338473 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67c47b2-2022-443f-b022-97152ab7472d None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:19.339006 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67c47b2-2022-443f-b022-97152ab7472d None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:19.342311 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67c47b2-2022-443f-b022-97152ab7472d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:19.358364 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-d67c47b2-2022-443f-b022-97152ab7472d None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:22:19.359686 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-d67c47b2-2022-443f-b022-97152ab7472d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:19.483232 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1025/2072] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:22:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 19:22:19.507184 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876f9086-b68c-4a62-9553-b9864a49cc99 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:19.509208 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-876f9086-b68c-4a62-9553-b9864a49cc99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:19.577098 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:22:19.591645 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876f9086-b68c-4a62-9553-b9864a49cc99 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:19.598691 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d52acf-2e33-4790-8a2b-215239ed387f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:19.600665 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d52acf-2e33-4790-8a2b-215239ed387f service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:19.601840 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d52acf-2e33-4790-8a2b-215239ed387f service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:19.602463 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d52acf-2e33-4790-8a2b-215239ed387f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:19.620066 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67c47b2-2022-443f-b022-97152ab7472d None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:19.621502 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-d67c47b2-2022-443f-b022-97152ab7472d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:19.625009 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d52acf-2e33-4790-8a2b-215239ed387f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:19.627114 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d52acf-2e33-4790-8a2b-215239ed387f service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:19.644275 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67c47b2-2022-443f-b022-97152ab7472d None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:19.646006 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1048/2073] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:22:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 02 19:22:19.650794 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016c4417-3c18-4365-b070-b4ce78b9bfbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:19.652110 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c4417-3c18-4365-b070-b4ce78b9bfbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:19.652868 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c4417-3c18-4365-b070-b4ce78b9bfbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:19.653276 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c4417-3c18-4365-b070-b4ce78b9bfbc admin admin] PATH_INFO: `/v3/projects/912c169ebc1a40ca96b4e31aab39435f` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:19.655652 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016c4417-3c18-4365-b070-b4ce78b9bfbc admin admin] RBAC: Authorizing `identity:delete_project(project_id=912c169ebc1a40ca96b4e31aab39435f)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:19.658020 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016c4417-3c18-4365-b070-b4ce78b9bfbc admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:19.806737 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-016c4417-3c18-4365-b070-b4ce78b9bfbc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '912c169ebc1a40ca96b4e31aab39435f', 'request_id': 'req-016c4417-3c18-4365-b070-b4ce78b9bfbc'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:19.898079 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-016c4417-3c18-4365-b070-b4ce78b9bfbc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '912c169ebc1a40ca96b4e31aab39435f', 'request_id': 'req-016c4417-3c18-4365-b070-b4ce78b9bfbc'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:19.908280 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-016c4417-3c18-4365-b070-b4ce78b9bfbc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '912c169ebc1a40ca96b4e31aab39435f', 'request_id': 'req-016c4417-3c18-4365-b070-b4ce78b9bfbc'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:19.926966 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-016c4417-3c18-4365-b070-b4ce78b9bfbc admin admin] The token cache is being invalidate because project 912c169ebc1a40ca96b4e31aab39435f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:22:19.928312 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-016c4417-3c18-4365-b070-b4ce78b9bfbc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:19.929455 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-016c4417-3c18-4365-b070-b4ce78b9bfbc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:19.936968 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1026/2074] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:22:19 2022] DELETE /identity/v3/projects/912c169ebc1a40ca96b4e31aab39435f => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:22:19.938926 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d52acf-2e33-4790-8a2b-215239ed387f service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:19.963405 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d52acf-2e33-4790-8a2b-215239ed387f service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:20.141126 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3de44a4-cfd9-45bf-bac3-c535da0d8ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:20.142036 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3de44a4-cfd9-45bf-bac3-c535da0d8ea9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:20.142363 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3de44a4-cfd9-45bf-bac3-c535da0d8ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:20.142590 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3de44a4-cfd9-45bf-bac3-c535da0d8ea9 admin admin] PATH_INFO: `/v3/users/37421de1be0446d4a9be4452aed13b9b` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:20.145431 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3de44a4-cfd9-45bf-bac3-c535da0d8ea9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37421de1be0446d4a9be4452aed13b9b)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:20.149366 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3de44a4-cfd9-45bf-bac3-c535da0d8ea9 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:20.269590 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016c4417-3c18-4365-b070-b4ce78b9bfbc admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:20.273342 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-016c4417-3c18-4365-b070-b4ce78b9bfbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:20.307802 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016c4417-3c18-4365-b070-b4ce78b9bfbc admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:20.321437 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762b2f4b-b6da-4fcb-86d9-e9aa23b4348b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:20.322431 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762b2f4b-b6da-4fcb-86d9-e9aa23b4348b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:20.322722 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762b2f4b-b6da-4fcb-86d9-e9aa23b4348b admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:20.323191 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762b2f4b-b6da-4fcb-86d9-e9aa23b4348b admin admin] PATH_INFO: `/v3/projects/8c71f8729510465b9868464021c28d3c` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:20.324862 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762b2f4b-b6da-4fcb-86d9-e9aa23b4348b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c71f8729510465b9868464021c28d3c)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:20.326916 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762b2f4b-b6da-4fcb-86d9-e9aa23b4348b admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:20.368144 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:22:20.425715 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-a3de44a4-cfd9-45bf-bac3-c535da0d8ea9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37421de1be0446d4a9be4452aed13b9b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:20.433450 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-a3de44a4-cfd9-45bf-bac3-c535da0d8ea9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37421de1be0446d4a9be4452aed13b9b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:20.468942 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-762b2f4b-b6da-4fcb-86d9-e9aa23b4348b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c71f8729510465b9868464021c28d3c', 'request_id': 'req-762b2f4b-b6da-4fcb-86d9-e9aa23b4348b'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:20.525979 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-a3de44a4-cfd9-45bf-bac3-c535da0d8ea9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37421de1be0446d4a9be4452aed13b9b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:20.527629 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-a3de44a4-cfd9-45bf-bac3-c535da0d8ea9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37421de1be0446d4a9be4452aed13b9b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:20.528350 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-a3de44a4-cfd9-45bf-bac3-c535da0d8ea9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37421de1be0446d4a9be4452aed13b9b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:20.550592 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1049/2075] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:22:19 2022] DELETE /identity/v3/users/37421de1be0446d4a9be4452aed13b9b => generated 0 bytes in 615 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:22:20.601416 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-762b2f4b-b6da-4fcb-86d9-e9aa23b4348b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c71f8729510465b9868464021c28d3c', 'request_id': 'req-762b2f4b-b6da-4fcb-86d9-e9aa23b4348b'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:20.612598 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-762b2f4b-b6da-4fcb-86d9-e9aa23b4348b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c71f8729510465b9868464021c28d3c', 'request_id': 'req-762b2f4b-b6da-4fcb-86d9-e9aa23b4348b'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:20.632128 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-762b2f4b-b6da-4fcb-86d9-e9aa23b4348b admin admin] The token cache is being invalidate because project 8c71f8729510465b9868464021c28d3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:22:20.633802 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-762b2f4b-b6da-4fcb-86d9-e9aa23b4348b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:20.635689 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-762b2f4b-b6da-4fcb-86d9-e9aa23b4348b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:20.638659 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1027/2076] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:22:20 2022] DELETE /identity/v3/projects/8c71f8729510465b9868464021c28d3c => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:22:20.642665 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3de44a4-cfd9-45bf-bac3-c535da0d8ea9 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:20.644346 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-a3de44a4-cfd9-45bf-bac3-c535da0d8ea9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:20.747258 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3de44a4-cfd9-45bf-bac3-c535da0d8ea9 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:20.752670 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d1a71d-a3fd-4048-9157-1e2d9078b23a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:20.754009 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d1a71d-a3fd-4048-9157-1e2d9078b23a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:20.754529 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d1a71d-a3fd-4048-9157-1e2d9078b23a admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:20.755070 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d1a71d-a3fd-4048-9157-1e2d9078b23a admin admin] PATH_INFO: `/v3/users/9e43e8fa3b1747b3942cec0dd05e882b` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:20.757186 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d1a71d-a3fd-4048-9157-1e2d9078b23a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e43e8fa3b1747b3942cec0dd05e882b)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:20.759577 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d1a71d-a3fd-4048-9157-1e2d9078b23a admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:20.766127 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:22:20.983713 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-b5d1a71d-a3fd-4048-9157-1e2d9078b23a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e43e8fa3b1747b3942cec0dd05e882b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:20.994789 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-b5d1a71d-a3fd-4048-9157-1e2d9078b23a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e43e8fa3b1747b3942cec0dd05e882b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:21.050212 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762b2f4b-b6da-4fcb-86d9-e9aa23b4348b admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:21.050212 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-762b2f4b-b6da-4fcb-86d9-e9aa23b4348b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:21.106354 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-b5d1a71d-a3fd-4048-9157-1e2d9078b23a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e43e8fa3b1747b3942cec0dd05e882b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:21.106354 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-b5d1a71d-a3fd-4048-9157-1e2d9078b23a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e43e8fa3b1747b3942cec0dd05e882b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:21.106354 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-b5d1a71d-a3fd-4048-9157-1e2d9078b23a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e43e8fa3b1747b3942cec0dd05e882b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:21.141996 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1050/2077] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:22:20 2022] DELETE /identity/v3/users/9e43e8fa3b1747b3942cec0dd05e882b => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 02 19:22:21.179324 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762b2f4b-b6da-4fcb-86d9-e9aa23b4348b admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:21.264291 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac7fa3d-efe4-4369-aeaf-2e5c1ea394f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:21.265686 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac7fa3d-efe4-4369-aeaf-2e5c1ea394f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:21.265979 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac7fa3d-efe4-4369-aeaf-2e5c1ea394f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:21.266262 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac7fa3d-efe4-4369-aeaf-2e5c1ea394f5 admin admin] PATH_INFO: `/v3/projects/56234860edf24fcfa3784c7e29e3d3e5` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:21.268001 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac7fa3d-efe4-4369-aeaf-2e5c1ea394f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56234860edf24fcfa3784c7e29e3d3e5)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:21.270231 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac7fa3d-efe4-4369-aeaf-2e5c1ea394f5 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:21.412902 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-5ac7fa3d-efe4-4369-aeaf-2e5c1ea394f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56234860edf24fcfa3784c7e29e3d3e5', 'request_id': 'req-5ac7fa3d-efe4-4369-aeaf-2e5c1ea394f5'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:21.645474 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-5ac7fa3d-efe4-4369-aeaf-2e5c1ea394f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56234860edf24fcfa3784c7e29e3d3e5', 'request_id': 'req-5ac7fa3d-efe4-4369-aeaf-2e5c1ea394f5'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:21.647509 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:22:21.652936 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-5ac7fa3d-efe4-4369-aeaf-2e5c1ea394f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56234860edf24fcfa3784c7e29e3d3e5', 'request_id': 'req-5ac7fa3d-efe4-4369-aeaf-2e5c1ea394f5'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:21.657924 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d1a71d-a3fd-4048-9157-1e2d9078b23a admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:21.659437 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-b5d1a71d-a3fd-4048-9157-1e2d9078b23a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:21.673519 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-5ac7fa3d-efe4-4369-aeaf-2e5c1ea394f5 admin admin] The token cache is being invalidate because project 56234860edf24fcfa3784c7e29e3d3e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:22:21.674007 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-5ac7fa3d-efe4-4369-aeaf-2e5c1ea394f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:21.674796 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-5ac7fa3d-efe4-4369-aeaf-2e5c1ea394f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:21.677034 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1028/2078] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:22:21 2022] DELETE /identity/v3/projects/56234860edf24fcfa3784c7e29e3d3e5 => generated 0 bytes in 630 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:22:21.685733 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1c20f0-bdf5-4c21-8c53-a44b447f1c87 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:21.686346 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1c20f0-bdf5-4c21-8c53-a44b447f1c87 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:21.686881 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1c20f0-bdf5-4c21-8c53-a44b447f1c87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:21.691777 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d1a71d-a3fd-4048-9157-1e2d9078b23a admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:21.711119 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-bf1c20f0-bdf5-4c21-8c53-a44b447f1c87 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:22:21.714148 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-bf1c20f0-bdf5-4c21-8c53-a44b447f1c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:21.819187 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1029/2079] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:22:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 19:22:21.833433 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbcce4c-0b5d-4042-89cf-7764b5633744 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:21.839065 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbcce4c-0b5d-4042-89cf-7764b5633744 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:21.839065 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbcce4c-0b5d-4042-89cf-7764b5633744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:21.842600 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69242aa7-6bdc-40dc-ac3a-713596bbc241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:21.843892 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69242aa7-6bdc-40dc-ac3a-713596bbc241 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:21.844309 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69242aa7-6bdc-40dc-ac3a-713596bbc241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:21.845036 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69242aa7-6bdc-40dc-ac3a-713596bbc241 admin admin] PATH_INFO: `/v3/projects/f90fcb16902e4d899c254718007ba426` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:21.846451 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69242aa7-6bdc-40dc-ac3a-713596bbc241 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f90fcb16902e4d899c254718007ba426)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:21.849089 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69242aa7-6bdc-40dc-ac3a-713596bbc241 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:21.870171 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-fdbcce4c-0b5d-4042-89cf-7764b5633744 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:22:21.872079 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-fdbcce4c-0b5d-4042-89cf-7764b5633744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:21.882550 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1030/2080] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:22:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:22:21.887102 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbcce4c-0b5d-4042-89cf-7764b5633744 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:21.901027 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbcce4c-0b5d-4042-89cf-7764b5633744 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:21.908155 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac2a7b2-cca8-45dd-855f-dc67a649342c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:21.909603 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac2a7b2-cca8-45dd-855f-dc67a649342c admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:21.910133 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac2a7b2-cca8-45dd-855f-dc67a649342c admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:21.911274 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac2a7b2-cca8-45dd-855f-dc67a649342c admin admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:21.913557 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac2a7b2-cca8-45dd-855f-dc67a649342c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:22:21.914297 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac2a7b2-cca8-45dd-855f-dc67a649342c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:21.916228 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac2a7b2-cca8-45dd-855f-dc67a649342c admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:21.942090 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1031/2081] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 2 19:22:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:22:21.945606 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac2a7b2-cca8-45dd-855f-dc67a649342c admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:21.975341 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac2a7b2-cca8-45dd-855f-dc67a649342c admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:21.979737 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5334f2-c39b-4ef9-ad45-eb7b20a2032b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:21.980476 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5334f2-c39b-4ef9-ad45-eb7b20a2032b admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:21.980774 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5334f2-c39b-4ef9-ad45-eb7b20a2032b admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:21.981635 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5334f2-c39b-4ef9-ad45-eb7b20a2032b admin admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:21.996839 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5334f2-c39b-4ef9-ad45-eb7b20a2032b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:22.002200 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5334f2-c39b-4ef9-ad45-eb7b20a2032b admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:22.101268 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-69242aa7-6bdc-40dc-ac3a-713596bbc241 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f90fcb16902e4d899c254718007ba426', 'request_id': 'req-69242aa7-6bdc-40dc-ac3a-713596bbc241'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:22.178163 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1032/2082] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:22:21 2022] POST /identity/v3/projects => generated 387 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:22:22.182452 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5334f2-c39b-4ef9-ad45-eb7b20a2032b admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:22.188110 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-69242aa7-6bdc-40dc-ac3a-713596bbc241 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f90fcb16902e4d899c254718007ba426', 'request_id': 'req-69242aa7-6bdc-40dc-ac3a-713596bbc241'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:22.200478 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-69242aa7-6bdc-40dc-ac3a-713596bbc241 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f90fcb16902e4d899c254718007ba426', 'request_id': 'req-69242aa7-6bdc-40dc-ac3a-713596bbc241'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:22.230351 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-69242aa7-6bdc-40dc-ac3a-713596bbc241 admin admin] The token cache is being invalidate because project f90fcb16902e4d899c254718007ba426 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:22:22.230699 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-69242aa7-6bdc-40dc-ac3a-713596bbc241 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:22.231414 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-69242aa7-6bdc-40dc-ac3a-713596bbc241 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:22.233103 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1051/2083] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:22:21 2022] DELETE /identity/v3/projects/f90fcb16902e4d899c254718007ba426 => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:22:22.237181 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69242aa7-6bdc-40dc-ac3a-713596bbc241 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:22.238989 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-69242aa7-6bdc-40dc-ac3a-713596bbc241 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:22.275144 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:22:22.315020 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5334f2-c39b-4ef9-ad45-eb7b20a2032b admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:22.339679 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4c9206-fdab-4e80-9eb2-97077c7e11bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:22.340586 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c9206-fdab-4e80-9eb2-97077c7e11bb admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:22.340870 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c9206-fdab-4e80-9eb2-97077c7e11bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:22.341132 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c9206-fdab-4e80-9eb2-97077c7e11bb admin admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:22.341931 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4c9206-fdab-4e80-9eb2-97077c7e11bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:22.343853 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4c9206-fdab-4e80-9eb2-97077c7e11bb admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:22.412633 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1033/2084] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:22:22 2022] POST /identity/v3/users => generated 294 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:22:22.420053 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4c9206-fdab-4e80-9eb2-97077c7e11bb admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:22.421947 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-9e4c9206-fdab-4e80-9eb2-97077c7e11bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:22.426577 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69242aa7-6bdc-40dc-ac3a-713596bbc241 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:22.436749 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4c9206-fdab-4e80-9eb2-97077c7e11bb admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:22.449043 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effe0497-a890-4adc-a4aa-e4a4a73d3615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:22.451554 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd3721a-108b-47ad-b0da-4b53077adb19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:22.451784 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effe0497-a890-4adc-a4aa-e4a4a73d3615 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:22.451784 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effe0497-a890-4adc-a4aa-e4a4a73d3615 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:22.451924 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3721a-108b-47ad-b0da-4b53077adb19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:22.452043 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effe0497-a890-4adc-a4aa-e4a4a73d3615 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:22.452181 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3721a-108b-47ad-b0da-4b53077adb19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:22.452181 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3721a-108b-47ad-b0da-4b53077adb19 admin admin] PATH_INFO: `/v3/users/b4dd78d14b4144b2b4245f16480459aa` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:22.452346 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effe0497-a890-4adc-a4aa-e4a4a73d3615 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:22.454437 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd3721a-108b-47ad-b0da-4b53077adb19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4dd78d14b4144b2b4245f16480459aa)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:22.454634 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effe0497-a890-4adc-a4aa-e4a4a73d3615 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:22.456480 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd3721a-108b-47ad-b0da-4b53077adb19 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:22.491751 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1034/2085] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:22:22 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:22:22.502156 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effe0497-a890-4adc-a4aa-e4a4a73d3615 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:22.514507 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effe0497-a890-4adc-a4aa-e4a4a73d3615 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:22.520602 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9fd423-6319-4538-92f6-04c7286c3521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:22.522251 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9fd423-6319-4538-92f6-04c7286c3521 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:22.522826 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9fd423-6319-4538-92f6-04c7286c3521 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:22.523291 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9fd423-6319-4538-92f6-04c7286c3521 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:22.524250 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9fd423-6319-4538-92f6-04c7286c3521 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:22.526549 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9fd423-6319-4538-92f6-04c7286c3521 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:22.539890 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1035/2086] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:22:22 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:22:22.546104 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9fd423-6319-4538-92f6-04c7286c3521 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:22.558261 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9fd423-6319-4538-92f6-04c7286c3521 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:22.563936 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585ca977-e270-449d-b296-09647f8bb8f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:22.565052 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585ca977-e270-449d-b296-09647f8bb8f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:22.565480 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585ca977-e270-449d-b296-09647f8bb8f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:22.565898 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585ca977-e270-449d-b296-09647f8bb8f7 admin admin] PATH_INFO: `/v3/projects/8b2e5841a73940008108e9e0283c359e/users/620c997f23b54bd6903d4aa26abc115d/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:22.582964 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585ca977-e270-449d-b296-09647f8bb8f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b2e5841a73940008108e9e0283c359e, user_id=620c997f23b54bd6903d4aa26abc115d, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:22.586050 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585ca977-e270-449d-b296-09647f8bb8f7 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:22.670935 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1036/2087] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:22:22 2022] PUT /identity/v3/projects/8b2e5841a73940008108e9e0283c359e/users/620c997f23b54bd6903d4aa26abc115d/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:22:22.682758 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d34b5-dddf-4b9f-81aa-870af031b2d7 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:22.683320 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d34b5-dddf-4b9f-81aa-870af031b2d7 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:22.683457 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d34b5-dddf-4b9f-81aa-870af031b2d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:22.711642 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-462d34b5-dddf-4b9f-81aa-870af031b2d7 None None] MFA Rules not processed for user `620c997f23b54bd6903d4aa26abc115d`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:22:22.712878 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-462d34b5-dddf-4b9f-81aa-870af031b2d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:22.780536 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:22:22.784673 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1037/2088] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:22:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:22:22.789922 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462d34b5-dddf-4b9f-81aa-870af031b2d7 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:22.795796 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-5cd3721a-108b-47ad-b0da-4b53077adb19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4dd78d14b4144b2b4245f16480459aa'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:22.804313 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-5cd3721a-108b-47ad-b0da-4b53077adb19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4dd78d14b4144b2b4245f16480459aa'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:22.862890 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462d34b5-dddf-4b9f-81aa-870af031b2d7 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:22.868163 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6802cb00-4fec-4994-a695-70068c3dbf32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:22.868845 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6802cb00-4fec-4994-a695-70068c3dbf32 admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:22.869272 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6802cb00-4fec-4994-a695-70068c3dbf32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:22.872435 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6802cb00-4fec-4994-a695-70068c3dbf32 admin admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:22.873530 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6802cb00-4fec-4994-a695-70068c3dbf32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:22.875365 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6802cb00-4fec-4994-a695-70068c3dbf32 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:22.879130 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-5cd3721a-108b-47ad-b0da-4b53077adb19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4dd78d14b4144b2b4245f16480459aa'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:22.879804 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-5cd3721a-108b-47ad-b0da-4b53077adb19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4dd78d14b4144b2b4245f16480459aa'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:22.880504 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-5cd3721a-108b-47ad-b0da-4b53077adb19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4dd78d14b4144b2b4245f16480459aa'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:22.903924 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1052/2089] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:22:22 2022] DELETE /identity/v3/users/b4dd78d14b4144b2b4245f16480459aa => generated 0 bytes in 664 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 02 19:22:22.904893 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd3721a-108b-47ad-b0da-4b53077adb19 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:22.906282 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-5cd3721a-108b-47ad-b0da-4b53077adb19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:22.970549 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1038/2090] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:22:22 2022] POST /identity/v3/projects => generated 385 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:22:22.976251 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6802cb00-4fec-4994-a695-70068c3dbf32 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:22.978360 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-6802cb00-4fec-4994-a695-70068c3dbf32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:23.001401 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd3721a-108b-47ad-b0da-4b53077adb19 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:23.072498 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6802cb00-4fec-4994-a695-70068c3dbf32 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:23.078186 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3f8c1b-4241-405a-a735-1b3db70939d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:23.078953 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3f8c1b-4241-405a-a735-1b3db70939d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:23.084594 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3f8c1b-4241-405a-a735-1b3db70939d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:23.084594 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3f8c1b-4241-405a-a735-1b3db70939d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:23.084594 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3f8c1b-4241-405a-a735-1b3db70939d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:23.084594 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3f8c1b-4241-405a-a735-1b3db70939d1 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:23.086287 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f307b089-8c72-4fa3-a146-9dd6829c6bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:23.087617 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f307b089-8c72-4fa3-a146-9dd6829c6bb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:23.087818 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f307b089-8c72-4fa3-a146-9dd6829c6bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:23.088326 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f307b089-8c72-4fa3-a146-9dd6829c6bb6 admin admin] PATH_INFO: `/v3/users/c587274f4a0945abaf3a4ac48f3fed29` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:23.089956 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f307b089-8c72-4fa3-a146-9dd6829c6bb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c587274f4a0945abaf3a4ac48f3fed29)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:23.092263 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f307b089-8c72-4fa3-a146-9dd6829c6bb6 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:23.134672 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:22:23.201298 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1039/2091] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:22:22 2022] POST /identity/v3/users => generated 299 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:22:23.202548 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3f8c1b-4241-405a-a735-1b3db70939d1 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:23.216896 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3f8c1b-4241-405a-a735-1b3db70939d1 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:23.222209 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778d4747-c05d-4d79-a097-2c00feba8e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:23.223229 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d4747-c05d-4d79-a097-2c00feba8e9b admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:23.225676 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d4747-c05d-4d79-a097-2c00feba8e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:23.225676 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d4747-c05d-4d79-a097-2c00feba8e9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:23.225676 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778d4747-c05d-4d79-a097-2c00feba8e9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:23.227508 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778d4747-c05d-4d79-a097-2c00feba8e9b admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:23.242471 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1040/2092] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:22:23 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:22:23.243698 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778d4747-c05d-4d79-a097-2c00feba8e9b admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:23.254446 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778d4747-c05d-4d79-a097-2c00feba8e9b admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:23.260119 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903569fc-3f1b-4d6b-b720-6bc22cb30bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:23.261797 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903569fc-3f1b-4d6b-b720-6bc22cb30bfc admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:23.261797 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903569fc-3f1b-4d6b-b720-6bc22cb30bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:23.261797 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903569fc-3f1b-4d6b-b720-6bc22cb30bfc admin admin] PATH_INFO: `/v3/projects/f2ca7ef4a018406997abc1f6703fd852/users/1319326c57094c349c4542d96d316b53/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:23.280832 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903569fc-3f1b-4d6b-b720-6bc22cb30bfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2ca7ef4a018406997abc1f6703fd852, user_id=1319326c57094c349c4542d96d316b53, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:23.285908 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903569fc-3f1b-4d6b-b720-6bc22cb30bfc admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:23.327569 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f307b089-8c72-4fa3-a146-9dd6829c6bb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c587274f4a0945abaf3a4ac48f3fed29'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:23.341793 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f307b089-8c72-4fa3-a146-9dd6829c6bb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c587274f4a0945abaf3a4ac48f3fed29'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:23.344482 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1041/2093] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:22:23 2022] PUT /identity/v3/projects/f2ca7ef4a018406997abc1f6703fd852/users/1319326c57094c349c4542d96d316b53/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:22:23.348734 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903569fc-3f1b-4d6b-b720-6bc22cb30bfc admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:23.404953 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f307b089-8c72-4fa3-a146-9dd6829c6bb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c587274f4a0945abaf3a4ac48f3fed29'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:23.406808 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f307b089-8c72-4fa3-a146-9dd6829c6bb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c587274f4a0945abaf3a4ac48f3fed29'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:23.407539 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f307b089-8c72-4fa3-a146-9dd6829c6bb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c587274f4a0945abaf3a4ac48f3fed29'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:23.432202 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1053/2094] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:22:22 2022] DELETE /identity/v3/users/c587274f4a0945abaf3a4ac48f3fed29 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:22:23.433671 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea64f15c-20ff-498c-b2c2-e49764b0c875 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:23.433898 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea64f15c-20ff-498c-b2c2-e49764b0c875 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:23.434184 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea64f15c-20ff-498c-b2c2-e49764b0c875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:23.445307 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903569fc-3f1b-4d6b-b720-6bc22cb30bfc admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:23.464158 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-ea64f15c-20ff-498c-b2c2-e49764b0c875 None None] MFA Rules not processed for user `85a9b91215d34c68827144968041e700`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:22:23.465405 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-ea64f15c-20ff-498c-b2c2-e49764b0c875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:23.491727 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:22:23.530869 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5ec4cd-5272-4d04-a133-aac07a7bd757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:23.530869 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ec4cd-5272-4d04-a133-aac07a7bd757 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:23.530869 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ec4cd-5272-4d04-a133-aac07a7bd757 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:23.531476 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ec4cd-5272-4d04-a133-aac07a7bd757 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:23.531727 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5ec4cd-5272-4d04-a133-aac07a7bd757 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:23.533346 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5ec4cd-5272-4d04-a133-aac07a7bd757 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:23.546937 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1042/2095] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:22:23 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:22:23.547389 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e058e93d-6c05-492a-bcf7-c0146c9bfb29 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:23.547855 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e058e93d-6c05-492a-bcf7-c0146c9bfb29 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:23.548141 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e058e93d-6c05-492a-bcf7-c0146c9bfb29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:23.570026 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-e058e93d-6c05-492a-bcf7-c0146c9bfb29 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:22:23.571508 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-e058e93d-6c05-492a-bcf7-c0146c9bfb29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:23.582089 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1043/2096] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:22:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:22:23.583085 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e058e93d-6c05-492a-bcf7-c0146c9bfb29 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:23.585227 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-e058e93d-6c05-492a-bcf7-c0146c9bfb29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:23.587745 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1054/2097] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:22:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:22:23.588841 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea64f15c-20ff-498c-b2c2-e49764b0c875 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:23.590708 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-ea64f15c-20ff-498c-b2c2-e49764b0c875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:23.606802 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e058e93d-6c05-492a-bcf7-c0146c9bfb29 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:23.610081 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea64f15c-20ff-498c-b2c2-e49764b0c875 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:23.613476 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dbcda2-8581-46de-b57c-90e6d739233d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:23.614413 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dbcda2-8581-46de-b57c-90e6d739233d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:23.615708 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dbcda2-8581-46de-b57c-90e6d739233d admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:23.615949 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab4baad-5eed-4935-8dd7-7fbac95fdcce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:23.616255 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dbcda2-8581-46de-b57c-90e6d739233d admin admin] PATH_INFO: `/v3/projects/5b70b7aa4377447b897f75e5dca91174` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:23.617107 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab4baad-5eed-4935-8dd7-7fbac95fdcce admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:23.617392 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab4baad-5eed-4935-8dd7-7fbac95fdcce admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:23.617712 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab4baad-5eed-4935-8dd7-7fbac95fdcce admin admin] PATH_INFO: `/v3/projects/f2ca7ef4a018406997abc1f6703fd852/users/1319326c57094c349c4542d96d316b53/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:23.617891 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dbcda2-8581-46de-b57c-90e6d739233d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b70b7aa4377447b897f75e5dca91174)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:23.619824 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab4baad-5eed-4935-8dd7-7fbac95fdcce admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2ca7ef4a018406997abc1f6703fd852, user_id=1319326c57094c349c4542d96d316b53, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:23.619955 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dbcda2-8581-46de-b57c-90e6d739233d admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:23.622814 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab4baad-5eed-4935-8dd7-7fbac95fdcce admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:23.636961 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1055/2098] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:22:23 2022] PUT /identity/v3/projects/f2ca7ef4a018406997abc1f6703fd852/users/1319326c57094c349c4542d96d316b53/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:22:23.638366 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab4baad-5eed-4935-8dd7-7fbac95fdcce admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:23.641125 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-fab4baad-5eed-4935-8dd7-7fbac95fdcce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:23.692564 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:22:23.710863 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab4baad-5eed-4935-8dd7-7fbac95fdcce admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:23.714716 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931dfddd-c5af-4615-a546-722885f2d7f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:23.715435 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931dfddd-c5af-4615-a546-722885f2d7f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:23.715758 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931dfddd-c5af-4615-a546-722885f2d7f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:23.716104 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931dfddd-c5af-4615-a546-722885f2d7f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:23.730539 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:INSERT=3,keystone:SELECT=93,keystone:DELETE=4 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:22:23.732150 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-30dbcda2-8581-46de-b57c-90e6d739233d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b70b7aa4377447b897f75e5dca91174', 'request_id': 'req-30dbcda2-8581-46de-b57c-90e6d739233d'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:23.797576 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931dfddd-c5af-4615-a546-722885f2d7f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:23.799135 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931dfddd-c5af-4615-a546-722885f2d7f5 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:23.799524 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-30dbcda2-8581-46de-b57c-90e6d739233d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b70b7aa4377447b897f75e5dca91174', 'request_id': 'req-30dbcda2-8581-46de-b57c-90e6d739233d'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:23.811562 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1056/2099] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:22:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:22:23.812164 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-30dbcda2-8581-46de-b57c-90e6d739233d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b70b7aa4377447b897f75e5dca91174', 'request_id': 'req-30dbcda2-8581-46de-b57c-90e6d739233d'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:23.820540 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931dfddd-c5af-4615-a546-722885f2d7f5 service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:23.831726 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-931dfddd-c5af-4615-a546-722885f2d7f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:23.834796 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-30dbcda2-8581-46de-b57c-90e6d739233d admin admin] The token cache is being invalidate because project 5b70b7aa4377447b897f75e5dca91174 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:22:23.835100 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-30dbcda2-8581-46de-b57c-90e6d739233d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:23.835806 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-30dbcda2-8581-46de-b57c-90e6d739233d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:23.841228 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1044/2100] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:22:23 2022] DELETE /identity/v3/projects/5b70b7aa4377447b897f75e5dca91174 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:22:23.855976 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a5f8e0-9f87-4403-bc22-c0529b14850c None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:23.856425 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a5f8e0-9f87-4403-bc22-c0529b14850c None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:23.856788 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a5f8e0-9f87-4403-bc22-c0529b14850c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:23.939713 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-98a5f8e0-9f87-4403-bc22-c0529b14850c None None] MFA Rules not processed for user `1319326c57094c349c4542d96d316b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:22:23.944566 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-98a5f8e0-9f87-4403-bc22-c0529b14850c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:23.951415 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931dfddd-c5af-4615-a546-722885f2d7f5 service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:24.064808 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b56dff1-83d4-4cfc-9735-cb9b82fbce29 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:24.065489 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b56dff1-83d4-4cfc-9735-cb9b82fbce29 service nova] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:24.065774 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b56dff1-83d4-4cfc-9735-cb9b82fbce29 service nova] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:24.066053 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b56dff1-83d4-4cfc-9735-cb9b82fbce29 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:24.068021 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-3b56dff1-83d4-4cfc-9735-cb9b82fbce29 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:24.077409 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1045/2101] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:22:23 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:22:24.078914 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b09865-9db5-46df-820e-f21c19bf08bc None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:24.079209 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b09865-9db5-46df-820e-f21c19bf08bc None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:24.079493 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b09865-9db5-46df-820e-f21c19bf08bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:24.095857 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:22:24.123411 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-57b09865-9db5-46df-820e-f21c19bf08bc None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:22:24.125033 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-57b09865-9db5-46df-820e-f21c19bf08bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:24.168058 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b56dff1-83d4-4cfc-9735-cb9b82fbce29 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:24.170900 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b56dff1-83d4-4cfc-9735-cb9b82fbce29 service nova] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:24.185759 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1057/2102] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:22:23 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:22:24.191939 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b56dff1-83d4-4cfc-9735-cb9b82fbce29 service nova] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:24.196009 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-3b56dff1-83d4-4cfc-9735-cb9b82fbce29 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:24.230887 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1046/2103] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:22:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:22:24.238644 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5088f96a-f471-44c0-85e8-0b29e13ce29e None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:24.238966 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5088f96a-f471-44c0-85e8-0b29e13ce29e None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:24.239252 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5088f96a-f471-44c0-85e8-0b29e13ce29e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:24.269285 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-5088f96a-f471-44c0-85e8-0b29e13ce29e None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:22:24.270832 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-5088f96a-f471-44c0-85e8-0b29e13ce29e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:24.285541 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1047/2104] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:22:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 19:22:24.286505 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5088f96a-f471-44c0-85e8-0b29e13ce29e None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:24.299748 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5088f96a-f471-44c0-85e8-0b29e13ce29e None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:24.306356 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872f527c-212d-4ae9-86cb-9aff949b3ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:24.307885 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872f527c-212d-4ae9-86cb-9aff949b3ff9 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:24.307885 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872f527c-212d-4ae9-86cb-9aff949b3ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:24.307885 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872f527c-212d-4ae9-86cb-9aff949b3ff9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:24.308604 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872f527c-212d-4ae9-86cb-9aff949b3ff9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:22:24.308973 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872f527c-212d-4ae9-86cb-9aff949b3ff9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:24.310688 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872f527c-212d-4ae9-86cb-9aff949b3ff9 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:24.317044 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b56dff1-83d4-4cfc-9735-cb9b82fbce29 service nova] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:24.322543 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72bef3b-e743-45cf-b1d4-4a039333a6ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:24.328808 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1048/2105] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 2 19:22:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:22:24.335315 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72bef3b-e743-45cf-b1d4-4a039333a6ba service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:24.335315 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72bef3b-e743-45cf-b1d4-4a039333a6ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:24.335315 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72bef3b-e743-45cf-b1d4-4a039333a6ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:24.335723 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872f527c-212d-4ae9-86cb-9aff949b3ff9 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:24.344027 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72bef3b-e743-45cf-b1d4-4a039333a6ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:24.345605 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872f527c-212d-4ae9-86cb-9aff949b3ff9 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:24.346605 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72bef3b-e743-45cf-b1d4-4a039333a6ba service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:24.352222 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c441ba3a-41f3-41d5-8161-36d9dbdaae96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:24.352973 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c441ba3a-41f3-41d5-8161-36d9dbdaae96 admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:24.353237 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c441ba3a-41f3-41d5-8161-36d9dbdaae96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:24.353568 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c441ba3a-41f3-41d5-8161-36d9dbdaae96 admin admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:24.354386 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c441ba3a-41f3-41d5-8161-36d9dbdaae96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:24.356683 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c441ba3a-41f3-41d5-8161-36d9dbdaae96 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:24.363578 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1058/2106] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:22:24 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:22:24.397793 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72bef3b-e743-45cf-b1d4-4a039333a6ba service cinder] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:24.403732 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-b72bef3b-e743-45cf-b1d4-4a039333a6ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:24.427753 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72bef3b-e743-45cf-b1d4-4a039333a6ba service cinder] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:24.432869 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae8263d-7a21-481f-8449-ce51416356d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:24.433930 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae8263d-7a21-481f-8449-ce51416356d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:24.434193 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae8263d-7a21-481f-8449-ce51416356d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:24.434478 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae8263d-7a21-481f-8449-ce51416356d4 admin admin] PATH_INFO: `/v3/projects/21706943ae514136899b35cec41bd931` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:24.435830 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae8263d-7a21-481f-8449-ce51416356d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21706943ae514136899b35cec41bd931)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:24.437884 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae8263d-7a21-481f-8449-ce51416356d4 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:24.453847 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1049/2107] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:22:24 2022] POST /identity/v3/projects => generated 369 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:22:24.457338 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c441ba3a-41f3-41d5-8161-36d9dbdaae96 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:24.471234 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-c441ba3a-41f3-41d5-8161-36d9dbdaae96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:24.505305 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:22:24.559062 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c441ba3a-41f3-41d5-8161-36d9dbdaae96 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:24.564825 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073be66c-13b7-492e-847e-a7c7bb063f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:24.565474 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073be66c-13b7-492e-847e-a7c7bb063f44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:24.565759 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073be66c-13b7-492e-847e-a7c7bb063f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:24.566040 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073be66c-13b7-492e-847e-a7c7bb063f44 admin admin] PATH_INFO: `/v3/users/620c997f23b54bd6903d4aa26abc115d` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:24.567541 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073be66c-13b7-492e-847e-a7c7bb063f44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=620c997f23b54bd6903d4aa26abc115d)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:24.569882 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073be66c-13b7-492e-847e-a7c7bb063f44 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:24.586911 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-6ae8263d-7a21-481f-8449-ce51416356d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21706943ae514136899b35cec41bd931', 'request_id': 'req-6ae8263d-7a21-481f-8449-ce51416356d4'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:24.679311 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-6ae8263d-7a21-481f-8449-ce51416356d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21706943ae514136899b35cec41bd931', 'request_id': 'req-6ae8263d-7a21-481f-8449-ce51416356d4'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:24.686424 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-6ae8263d-7a21-481f-8449-ce51416356d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21706943ae514136899b35cec41bd931', 'request_id': 'req-6ae8263d-7a21-481f-8449-ce51416356d4'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:24.700668 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-6ae8263d-7a21-481f-8449-ce51416356d4 admin admin] The token cache is being invalidate because project 21706943ae514136899b35cec41bd931 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:22:24.701187 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-6ae8263d-7a21-481f-8449-ce51416356d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:24.702353 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-6ae8263d-7a21-481f-8449-ce51416356d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:24.710305 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1059/2108] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:22:24 2022] DELETE /identity/v3/projects/21706943ae514136899b35cec41bd931 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 02 19:22:24.713538 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae8263d-7a21-481f-8449-ce51416356d4 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:24.718548 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-6ae8263d-7a21-481f-8449-ce51416356d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:24.827642 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:22:24.849826 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae8263d-7a21-481f-8449-ce51416356d4 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:24.856848 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7afa63-a2b7-41cd-b757-8565088dd592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:24.858438 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7afa63-a2b7-41cd-b757-8565088dd592 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:24.858954 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7afa63-a2b7-41cd-b757-8565088dd592 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:24.859335 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7afa63-a2b7-41cd-b757-8565088dd592 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:24.860529 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7afa63-a2b7-41cd-b757-8565088dd592 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:24.863191 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7afa63-a2b7-41cd-b757-8565088dd592 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:24.872637 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-073be66c-13b7-492e-847e-a7c7bb063f44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '620c997f23b54bd6903d4aa26abc115d'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:24.891461 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-073be66c-13b7-492e-847e-a7c7bb063f44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '620c997f23b54bd6903d4aa26abc115d'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:24.892604 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-073be66c-13b7-492e-847e-a7c7bb063f44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '620c997f23b54bd6903d4aa26abc115d'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:24.983295 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-073be66c-13b7-492e-847e-a7c7bb063f44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '620c997f23b54bd6903d4aa26abc115d'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:24.984245 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-073be66c-13b7-492e-847e-a7c7bb063f44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '620c997f23b54bd6903d4aa26abc115d'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:24.990347 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1060/2109] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:22:24 2022] POST /identity/v3/users => generated 285 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:22:24.995062 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7afa63-a2b7-41cd-b757-8565088dd592 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:24.998558 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-ee7afa63-a2b7-41cd-b757-8565088dd592 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:25.024988 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7afa63-a2b7-41cd-b757-8565088dd592 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:25.028333 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1050/2110] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:22:24 2022] DELETE /identity/v3/users/620c997f23b54bd6903d4aa26abc115d => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 02 19:22:25.035450 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b02033-1692-4bbc-83a0-9ec3c8aba32e None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:25.035752 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b02033-1692-4bbc-83a0-9ec3c8aba32e None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:25.036039 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b02033-1692-4bbc-83a0-9ec3c8aba32e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:25.056561 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-06b02033-1692-4bbc-83a0-9ec3c8aba32e None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:22:25.057850 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-06b02033-1692-4bbc-83a0-9ec3c8aba32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:25.130094 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681ef427-a7cb-4da5-8829-f4a3862a9f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:25.131716 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ef427-a7cb-4da5-8829-f4a3862a9f46 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:25.132006 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ef427-a7cb-4da5-8829-f4a3862a9f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:25.132284 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ef427-a7cb-4da5-8829-f4a3862a9f46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:25.133733 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681ef427-a7cb-4da5-8829-f4a3862a9f46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:25.138385 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681ef427-a7cb-4da5-8829-f4a3862a9f46 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:25.148935 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1051/2111] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:22:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:22:25.159020 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b02033-1692-4bbc-83a0-9ec3c8aba32e None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:25.164162 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1061/2112] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:22:24 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:22:25.165171 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681ef427-a7cb-4da5-8829-f4a3862a9f46 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:25.168706 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-681ef427-a7cb-4da5-8829-f4a3862a9f46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:25.183208 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b02033-1692-4bbc-83a0-9ec3c8aba32e None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:25.189371 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f1b0de-1ee7-4f05-a448-98a5f3230374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:25.190827 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f1b0de-1ee7-4f05-a448-98a5f3230374 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:25.191276 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f1b0de-1ee7-4f05-a448-98a5f3230374 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:25.191728 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f1b0de-1ee7-4f05-a448-98a5f3230374 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:25.192802 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f1b0de-1ee7-4f05-a448-98a5f3230374 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:25.195871 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f1b0de-1ee7-4f05-a448-98a5f3230374 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:25.207860 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1052/2113] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:22:25 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:22:25.212098 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f1b0de-1ee7-4f05-a448-98a5f3230374 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:25.235327 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f1b0de-1ee7-4f05-a448-98a5f3230374 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:25.241630 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baaa4ec-27cc-45b7-86f9-edf21b743f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:25.244213 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baaa4ec-27cc-45b7-86f9-edf21b743f59 admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:25.244681 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baaa4ec-27cc-45b7-86f9-edf21b743f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:25.245113 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baaa4ec-27cc-45b7-86f9-edf21b743f59 admin admin] PATH_INFO: `/v3/projects/0e3db6d00eed46f38a66fe513e0ac1a3/users/92dd1cba1c1648f39730f9560e781b10/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:25.268426 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baaa4ec-27cc-45b7-86f9-edf21b743f59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e3db6d00eed46f38a66fe513e0ac1a3, user_id=92dd1cba1c1648f39730f9560e781b10, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:25.273129 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681ef427-a7cb-4da5-8829-f4a3862a9f46 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:25.278060 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dc2988-7aae-4426-85e4-2d41d9f4635e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:25.278527 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc2988-7aae-4426-85e4-2d41d9f4635e service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:25.278853 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc2988-7aae-4426-85e4-2d41d9f4635e service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:25.279133 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc2988-7aae-4426-85e4-2d41d9f4635e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:25.281031 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baaa4ec-27cc-45b7-86f9-edf21b743f59 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:25.291749 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dc2988-7aae-4426-85e4-2d41d9f4635e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:25.293321 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dc2988-7aae-4426-85e4-2d41d9f4635e service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:25.305245 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1062/2114] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:22:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:22:25.349720 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1053/2115] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:22:25 2022] PUT /identity/v3/projects/0e3db6d00eed46f38a66fe513e0ac1a3/users/92dd1cba1c1648f39730f9560e781b10/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:22:25.363711 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b7fc77-5ec5-4c8c-ac01-27f59dbdf29b None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:25.364463 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b7fc77-5ec5-4c8c-ac01-27f59dbdf29b None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:25.364746 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b7fc77-5ec5-4c8c-ac01-27f59dbdf29b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:25.412336 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-b9b7fc77-5ec5-4c8c-ac01-27f59dbdf29b None None] MFA Rules not processed for user `92dd1cba1c1648f39730f9560e781b10`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:22:25.413636 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-b9b7fc77-5ec5-4c8c-ac01-27f59dbdf29b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:25.511429 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1063/2116] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:22:25 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:22:25.524318 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baaa4ec-27cc-45b7-86f9-edf21b743f59 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:25.526423 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-7baaa4ec-27cc-45b7-86f9-edf21b743f59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:25.533433 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:22:25.603938 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baaa4ec-27cc-45b7-86f9-edf21b743f59 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:25.608903 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850755e5-a9a7-483b-9834-7916fbeab5d8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07892c0aa21449b94a1cc78e3b8a653', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:25.610216 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850755e5-a9a7-483b-9834-7916fbeab5d8 service glance] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:25.610604 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850755e5-a9a7-483b-9834-7916fbeab5d8 service glance] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:25.611037 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850755e5-a9a7-483b-9834-7916fbeab5d8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:25.625052 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850755e5-a9a7-483b-9834-7916fbeab5d8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:25.628578 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850755e5-a9a7-483b-9834-7916fbeab5d8 service glance] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:25.647755 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1054/2117] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:22:25 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:22:25.900011 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b7fc77-5ec5-4c8c-ac01-27f59dbdf29b None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:25.901600 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-b9b7fc77-5ec5-4c8c-ac01-27f59dbdf29b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:25.911667 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:22:26.039688 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b7fc77-5ec5-4c8c-ac01-27f59dbdf29b None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:26.055956 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a5a188-95bf-419f-ad16-74cc32c00206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:26.057035 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5a188-95bf-419f-ad16-74cc32c00206 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:26.058028 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5a188-95bf-419f-ad16-74cc32c00206 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:26.058491 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5a188-95bf-419f-ad16-74cc32c00206 admin admin] PATH_INFO: `/v3/projects/8b2e5841a73940008108e9e0283c359e` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:26.060075 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a5a188-95bf-419f-ad16-74cc32c00206 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b2e5841a73940008108e9e0283c359e)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:26.063058 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a5a188-95bf-419f-ad16-74cc32c00206 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:26.237966 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f3a5a188-95bf-419f-ad16-74cc32c00206 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b2e5841a73940008108e9e0283c359e', 'request_id': 'req-f3a5a188-95bf-419f-ad16-74cc32c00206'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:26.365136 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f3a5a188-95bf-419f-ad16-74cc32c00206 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b2e5841a73940008108e9e0283c359e', 'request_id': 'req-f3a5a188-95bf-419f-ad16-74cc32c00206'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:26.372126 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f3a5a188-95bf-419f-ad16-74cc32c00206 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b2e5841a73940008108e9e0283c359e', 'request_id': 'req-f3a5a188-95bf-419f-ad16-74cc32c00206'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:26.387922 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f3a5a188-95bf-419f-ad16-74cc32c00206 admin admin] The token cache is being invalidate because project 8b2e5841a73940008108e9e0283c359e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:22:26.387922 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f3a5a188-95bf-419f-ad16-74cc32c00206 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:26.388425 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f3a5a188-95bf-419f-ad16-74cc32c00206 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:26.390405 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1064/2118] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:22:25 2022] DELETE /identity/v3/projects/8b2e5841a73940008108e9e0283c359e => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:22:26.394370 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850755e5-a9a7-483b-9834-7916fbeab5d8 service glance] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:26.406620 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850755e5-a9a7-483b-9834-7916fbeab5d8 service glance] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:26.506978 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93146d18-76c6-43aa-96ea-1a3b628e4bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:26.508064 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93146d18-76c6-43aa-96ea-1a3b628e4bb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:26.508457 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93146d18-76c6-43aa-96ea-1a3b628e4bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:26.508796 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93146d18-76c6-43aa-96ea-1a3b628e4bb2 admin admin] PATH_INFO: `/v3/users/1319326c57094c349c4542d96d316b53` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:26.510169 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93146d18-76c6-43aa-96ea-1a3b628e4bb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1319326c57094c349c4542d96d316b53)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:26.512351 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93146d18-76c6-43aa-96ea-1a3b628e4bb2 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:26.765399 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-93146d18-76c6-43aa-96ea-1a3b628e4bb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1319326c57094c349c4542d96d316b53'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:26.778041 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-93146d18-76c6-43aa-96ea-1a3b628e4bb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1319326c57094c349c4542d96d316b53'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:26.778857 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-93146d18-76c6-43aa-96ea-1a3b628e4bb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1319326c57094c349c4542d96d316b53'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:26.855965 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-93146d18-76c6-43aa-96ea-1a3b628e4bb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1319326c57094c349c4542d96d316b53'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:26.857102 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-93146d18-76c6-43aa-96ea-1a3b628e4bb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1319326c57094c349c4542d96d316b53'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:26.911409 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1055/2119] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:22:26 2022] DELETE /identity/v3/users/1319326c57094c349c4542d96d316b53 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:22:27.650263 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016844e4-60ce-49f5-b256-c75e89712454 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:27.650868 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016844e4-60ce-49f5-b256-c75e89712454 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:27.651005 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016844e4-60ce-49f5-b256-c75e89712454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:27.673785 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93146d18-76c6-43aa-96ea-1a3b628e4bb2 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:27.675439 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-93146d18-76c6-43aa-96ea-1a3b628e4bb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:27.692708 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-016844e4-60ce-49f5-b256-c75e89712454 None None] MFA Rules not processed for user `85a9b91215d34c68827144968041e700`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:22:27.697338 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-016844e4-60ce-49f5-b256-c75e89712454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:27.727336 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:22:27.835821 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93146d18-76c6-43aa-96ea-1a3b628e4bb2 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:27.837110 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1065/2120] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:22:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:22:27.845329 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a276977-5d3b-4390-893a-733786fd1f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:27.847100 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a276977-5d3b-4390-893a-733786fd1f6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:27.848519 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a276977-5d3b-4390-893a-733786fd1f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:27.848891 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a276977-5d3b-4390-893a-733786fd1f6a admin admin] PATH_INFO: `/v3/projects/f2ca7ef4a018406997abc1f6703fd852` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:27.851291 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a276977-5d3b-4390-893a-733786fd1f6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2ca7ef4a018406997abc1f6703fd852)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:27.855835 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a276977-5d3b-4390-893a-733786fd1f6a admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:28.018082 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-2a276977-5d3b-4390-893a-733786fd1f6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2ca7ef4a018406997abc1f6703fd852', 'request_id': 'req-2a276977-5d3b-4390-893a-733786fd1f6a'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:28.101652 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-2a276977-5d3b-4390-893a-733786fd1f6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2ca7ef4a018406997abc1f6703fd852', 'request_id': 'req-2a276977-5d3b-4390-893a-733786fd1f6a'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:28.123446 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-2a276977-5d3b-4390-893a-733786fd1f6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2ca7ef4a018406997abc1f6703fd852', 'request_id': 'req-2a276977-5d3b-4390-893a-733786fd1f6a'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:28.150529 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-2a276977-5d3b-4390-893a-733786fd1f6a admin admin] The token cache is being invalidate because project f2ca7ef4a018406997abc1f6703fd852 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:22:28.150529 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-2a276977-5d3b-4390-893a-733786fd1f6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:28.151179 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-2a276977-5d3b-4390-893a-733786fd1f6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:28.153320 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1056/2121] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:22:27 2022] DELETE /identity/v3/projects/f2ca7ef4a018406997abc1f6703fd852 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:22:28.158659 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d2b094-361f-4e3b-b8cd-ab6bba53b5c9 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:28.158999 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d2b094-361f-4e3b-b8cd-ab6bba53b5c9 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:28.159285 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d2b094-361f-4e3b-b8cd-ab6bba53b5c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:28.182725 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-93d2b094-361f-4e3b-b8cd-ab6bba53b5c9 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:22:28.185089 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-93d2b094-361f-4e3b-b8cd-ab6bba53b5c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:28.236660 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:22:28.272759 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1066/2122] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:22:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 19:22:28.280963 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd6356b-e7c1-49b2-8219-4e72c707f063 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:28.281249 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd6356b-e7c1-49b2-8219-4e72c707f063 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:28.281529 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd6356b-e7c1-49b2-8219-4e72c707f063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:28.303860 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-dbd6356b-e7c1-49b2-8219-4e72c707f063 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:22:28.305218 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-dbd6356b-e7c1-49b2-8219-4e72c707f063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:28.313928 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1057/2123] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:22:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:22:28.319100 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d2b094-361f-4e3b-b8cd-ab6bba53b5c9 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:28.330319 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d2b094-361f-4e3b-b8cd-ab6bba53b5c9 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:28.337896 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c36211b-64df-4119-9488-f417f5d39f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:28.338614 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c36211b-64df-4119-9488-f417f5d39f7b admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:28.338940 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c36211b-64df-4119-9488-f417f5d39f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:28.339374 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c36211b-64df-4119-9488-f417f5d39f7b admin admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:28.340201 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c36211b-64df-4119-9488-f417f5d39f7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:22:28.340517 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c36211b-64df-4119-9488-f417f5d39f7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:28.342169 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c36211b-64df-4119-9488-f417f5d39f7b admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:28.355523 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1067/2124] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 2 19:22:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:22:28.360574 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd6356b-e7c1-49b2-8219-4e72c707f063 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:28.371446 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd6356b-e7c1-49b2-8219-4e72c707f063 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:28.376319 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee5f3ba-284e-481f-a2b9-0a042b0b67c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:28.377126 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee5f3ba-284e-481f-a2b9-0a042b0b67c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:28.377416 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee5f3ba-284e-481f-a2b9-0a042b0b67c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:28.377741 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee5f3ba-284e-481f-a2b9-0a042b0b67c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:28.378586 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee5f3ba-284e-481f-a2b9-0a042b0b67c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:28.383477 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee5f3ba-284e-481f-a2b9-0a042b0b67c0 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:28.510161 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1058/2125] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:22:28 2022] POST /identity/v3/projects => generated 381 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:22:28.514322 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c36211b-64df-4119-9488-f417f5d39f7b admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:28.524809 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c36211b-64df-4119-9488-f417f5d39f7b admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:28.585700 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b230b8f3-9d1e-45eb-acba-70aed8e709a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:28.586384 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b230b8f3-9d1e-45eb-acba-70aed8e709a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:28.586764 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b230b8f3-9d1e-45eb-acba-70aed8e709a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:28.587017 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b230b8f3-9d1e-45eb-acba-70aed8e709a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:28.587716 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b230b8f3-9d1e-45eb-acba-70aed8e709a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:28.589641 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b230b8f3-9d1e-45eb-acba-70aed8e709a1 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:28.687555 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1068/2126] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:22:28 2022] POST /identity/v3/users => generated 291 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 02 19:22:28.690908 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee5f3ba-284e-481f-a2b9-0a042b0b67c0 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:28.705165 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee5f3ba-284e-481f-a2b9-0a042b0b67c0 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:28.717804 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709b8720-f427-4e78-8ebb-3753ca84997e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:28.718772 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709b8720-f427-4e78-8ebb-3753ca84997e admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:28.719248 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709b8720-f427-4e78-8ebb-3753ca84997e admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:28.719765 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709b8720-f427-4e78-8ebb-3753ca84997e admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:28.720467 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709b8720-f427-4e78-8ebb-3753ca84997e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:28.722102 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709b8720-f427-4e78-8ebb-3753ca84997e admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:28.735685 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1059/2127] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:22:28 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:22:28.739615 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b230b8f3-9d1e-45eb-acba-70aed8e709a1 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:28.751632 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b230b8f3-9d1e-45eb-acba-70aed8e709a1 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:28.757875 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a3ff56-7b1a-4683-a322-994aeee4ba0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:28.762967 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a3ff56-7b1a-4683-a322-994aeee4ba0a admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:28.762967 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a3ff56-7b1a-4683-a322-994aeee4ba0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:28.762967 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a3ff56-7b1a-4683-a322-994aeee4ba0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:28.762967 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a3ff56-7b1a-4683-a322-994aeee4ba0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:28.764338 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a3ff56-7b1a-4683-a322-994aeee4ba0a admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:28.783225 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709b8720-f427-4e78-8ebb-3753ca84997e admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:28.791212 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1069/2128] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:22:28 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:22:28.797798 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709b8720-f427-4e78-8ebb-3753ca84997e admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:28.805300 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bbf313-d112-438c-b9a9-2ee712011c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:28.806094 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bbf313-d112-438c-b9a9-2ee712011c34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:28.806377 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bbf313-d112-438c-b9a9-2ee712011c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:28.806652 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bbf313-d112-438c-b9a9-2ee712011c34 admin admin] PATH_INFO: `/v3/projects/c26b7ffc784349f2b905b804aaf568a5/users/2dd45764bb194bf6bf31607fad554a98/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:28.840338 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bbf313-d112-438c-b9a9-2ee712011c34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c26b7ffc784349f2b905b804aaf568a5, user_id=2dd45764bb194bf6bf31607fad554a98, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:28.841906 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bbf313-d112-438c-b9a9-2ee712011c34 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:28.892128 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1060/2129] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:22:28 2022] PUT /identity/v3/projects/c26b7ffc784349f2b905b804aaf568a5/users/2dd45764bb194bf6bf31607fad554a98/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:22:28.906302 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18294803-6fc7-45f6-96ea-c8bb698e0821 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:28.906597 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18294803-6fc7-45f6-96ea-c8bb698e0821 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:28.906996 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18294803-6fc7-45f6-96ea-c8bb698e0821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:28.934929 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-18294803-6fc7-45f6-96ea-c8bb698e0821 None None] MFA Rules not processed for user `2dd45764bb194bf6bf31607fad554a98`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:22:28.936192 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-18294803-6fc7-45f6-96ea-c8bb698e0821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:29.019088 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1070/2130] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:22:28 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:22:29.024831 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bbf313-d112-438c-b9a9-2ee712011c34 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:29.099449 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:22:29.125008 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bbf313-d112-438c-b9a9-2ee712011c34 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:29.131341 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0ea212-0c0c-4f70-8e10-26df647fd67c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:29.132020 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ea212-0c0c-4f70-8e10-26df647fd67c admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:29.132307 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ea212-0c0c-4f70-8e10-26df647fd67c admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:29.132590 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ea212-0c0c-4f70-8e10-26df647fd67c admin admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:29.133294 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0ea212-0c0c-4f70-8e10-26df647fd67c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:29.135426 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0ea212-0c0c-4f70-8e10-26df647fd67c admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:29.250228 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1061/2131] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:22:29 2022] POST /identity/v3/projects => generated 381 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:22:29.251763 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18294803-6fc7-45f6-96ea-c8bb698e0821 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:29.263211 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18294803-6fc7-45f6-96ea-c8bb698e0821 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:29.301049 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:22:29.328428 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b2b74d-f5b7-4d0c-8c62-415f7605f4b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:29.329126 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2b74d-f5b7-4d0c-8c62-415f7605f4b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:29.329408 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2b74d-f5b7-4d0c-8c62-415f7605f4b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:29.329685 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2b74d-f5b7-4d0c-8c62-415f7605f4b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:29.330593 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b2b74d-f5b7-4d0c-8c62-415f7605f4b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:29.332811 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b2b74d-f5b7-4d0c-8c62-415f7605f4b6 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:29.435490 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1071/2132] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:22:29 2022] POST /identity/v3/users => generated 297 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:22:29.440133 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0ea212-0c0c-4f70-8e10-26df647fd67c admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:29.452632 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0ea212-0c0c-4f70-8e10-26df647fd67c admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:29.457706 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449562be-88c9-4ebe-88e4-ebd4c0ff3824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:29.458204 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449562be-88c9-4ebe-88e4-ebd4c0ff3824 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:29.458498 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449562be-88c9-4ebe-88e4-ebd4c0ff3824 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:29.458835 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449562be-88c9-4ebe-88e4-ebd4c0ff3824 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:29.460127 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449562be-88c9-4ebe-88e4-ebd4c0ff3824 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:29.461814 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449562be-88c9-4ebe-88e4-ebd4c0ff3824 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:29.474525 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1062/2133] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:22:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:22:29.479025 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b2b74d-f5b7-4d0c-8c62-415f7605f4b6 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:29.489462 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b2b74d-f5b7-4d0c-8c62-415f7605f4b6 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:29.494311 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c538072e-4032-428b-94b6-b2486bff0e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:29.495308 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c538072e-4032-428b-94b6-b2486bff0e6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:29.495594 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c538072e-4032-428b-94b6-b2486bff0e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:29.495871 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c538072e-4032-428b-94b6-b2486bff0e6a admin admin] PATH_INFO: `/v3/projects/a254495b329146779d6841bb64c6af2d/users/1be611853d27436194ac5dbc3e0ccb8d/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:29.512151 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c538072e-4032-428b-94b6-b2486bff0e6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a254495b329146779d6841bb64c6af2d, user_id=1be611853d27436194ac5dbc3e0ccb8d, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:29.514767 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c538072e-4032-428b-94b6-b2486bff0e6a admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:29.623568 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1072/2134] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:22:29 2022] PUT /identity/v3/projects/a254495b329146779d6841bb64c6af2d/users/1be611853d27436194ac5dbc3e0ccb8d/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:22:29.627980 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449562be-88c9-4ebe-88e4-ebd4c0ff3824 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:29.638727 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449562be-88c9-4ebe-88e4-ebd4c0ff3824 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:29.698980 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3ce2d0-9dc3-4f7d-a9b7-036242de170c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:29.699669 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ce2d0-9dc3-4f7d-a9b7-036242de170c admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:29.700082 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ce2d0-9dc3-4f7d-a9b7-036242de170c admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:29.700371 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ce2d0-9dc3-4f7d-a9b7-036242de170c admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:29.701117 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3ce2d0-9dc3-4f7d-a9b7-036242de170c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:29.702684 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3ce2d0-9dc3-4f7d-a9b7-036242de170c admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:29.713143 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1063/2135] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:22:29 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 02 19:22:29.717685 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c538072e-4032-428b-94b6-b2486bff0e6a admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:29.729703 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c538072e-4032-428b-94b6-b2486bff0e6a admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:29.734903 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595b5285-cafb-4995-8b63-7b95acb6a4c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:29.735793 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595b5285-cafb-4995-8b63-7b95acb6a4c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:29.736084 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595b5285-cafb-4995-8b63-7b95acb6a4c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:29.736363 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595b5285-cafb-4995-8b63-7b95acb6a4c1 admin admin] PATH_INFO: `/v3/projects/a254495b329146779d6841bb64c6af2d/users/1be611853d27436194ac5dbc3e0ccb8d/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:29.738460 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595b5285-cafb-4995-8b63-7b95acb6a4c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a254495b329146779d6841bb64c6af2d, user_id=1be611853d27436194ac5dbc3e0ccb8d, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:29.741696 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595b5285-cafb-4995-8b63-7b95acb6a4c1 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:29.753212 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1073/2136] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:22:29 2022] PUT /identity/v3/projects/a254495b329146779d6841bb64c6af2d/users/1be611853d27436194ac5dbc3e0ccb8d/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:22:29.765428 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6493b0-c232-4095-8d0c-9ae924aa19ad None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:29.765659 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6493b0-c232-4095-8d0c-9ae924aa19ad None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:29.766077 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6493b0-c232-4095-8d0c-9ae924aa19ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:29.795164 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-2a6493b0-c232-4095-8d0c-9ae924aa19ad None None] MFA Rules not processed for user `1be611853d27436194ac5dbc3e0ccb8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:22:29.796384 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-2a6493b0-c232-4095-8d0c-9ae924aa19ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:29.876751 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1064/2137] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:22:29 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:22:29.888888 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595b5285-cafb-4995-8b63-7b95acb6a4c1 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:29.894889 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-595b5285-cafb-4995-8b63-7b95acb6a4c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:29.967052 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595b5285-cafb-4995-8b63-7b95acb6a4c1 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:29.970993 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963dd056-7186-4f8c-b1fe-b082cf69d01e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:29.971835 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963dd056-7186-4f8c-b1fe-b082cf69d01e service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:29.972119 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963dd056-7186-4f8c-b1fe-b082cf69d01e service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:29.972394 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963dd056-7186-4f8c-b1fe-b082cf69d01e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:29.988332 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963dd056-7186-4f8c-b1fe-b082cf69d01e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:29.989875 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963dd056-7186-4f8c-b1fe-b082cf69d01e service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:30.003181 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1074/2138] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:22:29 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:22:30.130673 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6493b0-c232-4095-8d0c-9ae924aa19ad None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:30.141138 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6493b0-c232-4095-8d0c-9ae924aa19ad None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:30.144780 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeb84e4-59f1-4178-a89a-15ae30f98f6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:30.145434 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeb84e4-59f1-4178-a89a-15ae30f98f6d service cinder] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:30.145724 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeb84e4-59f1-4178-a89a-15ae30f98f6d service cinder] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:30.146008 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeb84e4-59f1-4178-a89a-15ae30f98f6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:30.200838 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:22:30.213237 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeb84e4-59f1-4178-a89a-15ae30f98f6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:30.214793 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeb84e4-59f1-4178-a89a-15ae30f98f6d service cinder] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:30.225005 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1065/2139] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:22:30 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:22:33.439686 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963dd056-7186-4f8c-b1fe-b082cf69d01e service cinder] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:33.545003 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963dd056-7186-4f8c-b1fe-b082cf69d01e service cinder] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:33.549666 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a375977e-b7a2-4e30-821d-c04523f3500a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:33.550570 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a375977e-b7a2-4e30-821d-c04523f3500a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:33.550887 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a375977e-b7a2-4e30-821d-c04523f3500a admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:33.551167 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a375977e-b7a2-4e30-821d-c04523f3500a admin admin] PATH_INFO: `/v3/users/2dd45764bb194bf6bf31607fad554a98` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:33.552408 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a375977e-b7a2-4e30-821d-c04523f3500a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2dd45764bb194bf6bf31607fad554a98)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:33.554387 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a375977e-b7a2-4e30-821d-c04523f3500a admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:33.562566 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:22:33.787294 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-a375977e-b7a2-4e30-821d-c04523f3500a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2dd45764bb194bf6bf31607fad554a98'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:33.794395 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-a375977e-b7a2-4e30-821d-c04523f3500a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2dd45764bb194bf6bf31607fad554a98'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:33.874642 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-a375977e-b7a2-4e30-821d-c04523f3500a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2dd45764bb194bf6bf31607fad554a98'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:33.875465 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-a375977e-b7a2-4e30-821d-c04523f3500a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2dd45764bb194bf6bf31607fad554a98'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:33.876195 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-a375977e-b7a2-4e30-821d-c04523f3500a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2dd45764bb194bf6bf31607fad554a98'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:33.897906 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1075/2140] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:22:33 2022] DELETE /identity/v3/users/2dd45764bb194bf6bf31607fad554a98 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:22:33.902771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05366d76-483e-4dc0-ae61-82077f1163ab None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:33.903364 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05366d76-483e-4dc0-ae61-82077f1163ab None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:33.903364 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05366d76-483e-4dc0-ae61-82077f1163ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:33.974274 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-05366d76-483e-4dc0-ae61-82077f1163ab None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:22:33.975723 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-05366d76-483e-4dc0-ae61-82077f1163ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:34.090166 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1066/2141] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:22:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:22:34.098221 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a375977e-b7a2-4e30-821d-c04523f3500a admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:34.100040 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-a375977e-b7a2-4e30-821d-c04523f3500a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:34.258493 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a375977e-b7a2-4e30-821d-c04523f3500a admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:34.263300 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93599820-bbd7-46ee-af62-357ad9efd5c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:34.264095 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93599820-bbd7-46ee-af62-357ad9efd5c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:34.264455 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93599820-bbd7-46ee-af62-357ad9efd5c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:34.264826 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93599820-bbd7-46ee-af62-357ad9efd5c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:34.283588 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93599820-bbd7-46ee-af62-357ad9efd5c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:34.285616 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93599820-bbd7-46ee-af62-357ad9efd5c1 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:34.301685 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1076/2142] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:22:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 02 19:22:34.856959 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05366d76-483e-4dc0-ae61-82077f1163ab None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:34.858442 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-05366d76-483e-4dc0-ae61-82077f1163ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:34.871801 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05366d76-483e-4dc0-ae61-82077f1163ab None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:34.891112 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb975a4b-1b14-444c-bbc1-655df29ae3b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:34.891112 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb975a4b-1b14-444c-bbc1-655df29ae3b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:34.891112 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb975a4b-1b14-444c-bbc1-655df29ae3b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:34.891112 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb975a4b-1b14-444c-bbc1-655df29ae3b0 admin admin] PATH_INFO: `/v3/projects/c26b7ffc784349f2b905b804aaf568a5` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:34.892358 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb975a4b-1b14-444c-bbc1-655df29ae3b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c26b7ffc784349f2b905b804aaf568a5)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:34.892358 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb975a4b-1b14-444c-bbc1-655df29ae3b0 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:35.007192 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-cb975a4b-1b14-444c-bbc1-655df29ae3b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c26b7ffc784349f2b905b804aaf568a5', 'request_id': 'req-cb975a4b-1b14-444c-bbc1-655df29ae3b0'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:35.107561 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-cb975a4b-1b14-444c-bbc1-655df29ae3b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c26b7ffc784349f2b905b804aaf568a5', 'request_id': 'req-cb975a4b-1b14-444c-bbc1-655df29ae3b0'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:35.114472 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-cb975a4b-1b14-444c-bbc1-655df29ae3b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c26b7ffc784349f2b905b804aaf568a5', 'request_id': 'req-cb975a4b-1b14-444c-bbc1-655df29ae3b0'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:35.133695 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-cb975a4b-1b14-444c-bbc1-655df29ae3b0 admin admin] The token cache is being invalidate because project c26b7ffc784349f2b905b804aaf568a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:22:35.134003 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-cb975a4b-1b14-444c-bbc1-655df29ae3b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:35.134766 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-cb975a4b-1b14-444c-bbc1-655df29ae3b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:35.136516 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1067/2143] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:22:34 2022] DELETE /identity/v3/projects/c26b7ffc784349f2b905b804aaf568a5 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:22:35.140565 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93599820-bbd7-46ee-af62-357ad9efd5c1 service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:35.155057 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93599820-bbd7-46ee-af62-357ad9efd5c1 service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:35.205843 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:22:35.215300 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48843d77-cc21-493b-b14f-ece45da34846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:35.216191 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48843d77-cc21-493b-b14f-ece45da34846 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:35.216476 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48843d77-cc21-493b-b14f-ece45da34846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:35.216761 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48843d77-cc21-493b-b14f-ece45da34846 admin admin] PATH_INFO: `/v3/users/1be611853d27436194ac5dbc3e0ccb8d` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:35.217980 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48843d77-cc21-493b-b14f-ece45da34846 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1be611853d27436194ac5dbc3e0ccb8d)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:35.219968 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48843d77-cc21-493b-b14f-ece45da34846 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:35.431658 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-48843d77-cc21-493b-b14f-ece45da34846 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1be611853d27436194ac5dbc3e0ccb8d'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:35.441556 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-48843d77-cc21-493b-b14f-ece45da34846 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1be611853d27436194ac5dbc3e0ccb8d'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:35.536787 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-48843d77-cc21-493b-b14f-ece45da34846 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1be611853d27436194ac5dbc3e0ccb8d'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:35.539274 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-48843d77-cc21-493b-b14f-ece45da34846 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1be611853d27436194ac5dbc3e0ccb8d'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:35.539274 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-48843d77-cc21-493b-b14f-ece45da34846 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1be611853d27436194ac5dbc3e0ccb8d'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:35.568773 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1077/2144] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:22:35 2022] DELETE /identity/v3/users/1be611853d27436194ac5dbc3e0ccb8d => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:22:36.118212 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37de444-dee9-4663-8e62-d3b42942ee1f None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:36.118696 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37de444-dee9-4663-8e62-d3b42942ee1f None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:36.118830 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37de444-dee9-4663-8e62-d3b42942ee1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:36.148795 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-e37de444-dee9-4663-8e62-d3b42942ee1f None None] MFA Rules not processed for user `04f64813f09348eb91cb7bb2869f7612`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:22:36.150053 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-e37de444-dee9-4663-8e62-d3b42942ee1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:36.169050 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48843d77-cc21-493b-b14f-ece45da34846 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:36.171034 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-48843d77-cc21-493b-b14f-ece45da34846 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:36.208520 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:22:36.226279 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1068/2145] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Jun 2 19:22:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:22:36.298428 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48843d77-cc21-493b-b14f-ece45da34846 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:36.305617 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b1dc50-4a68-495a-8aa6-da3a38ace17e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:36.306695 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37de444-dee9-4663-8e62-d3b42942ee1f None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:36.306896 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b1dc50-4a68-495a-8aa6-da3a38ace17e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:36.307240 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b1dc50-4a68-495a-8aa6-da3a38ace17e admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:36.307594 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b1dc50-4a68-495a-8aa6-da3a38ace17e admin admin] PATH_INFO: `/v3/projects/a254495b329146779d6841bb64c6af2d` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:36.308969 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-e37de444-dee9-4663-8e62-d3b42942ee1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:36.309197 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b1dc50-4a68-495a-8aa6-da3a38ace17e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a254495b329146779d6841bb64c6af2d)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:36.311371 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b1dc50-4a68-495a-8aa6-da3a38ace17e admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:36.414105 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37de444-dee9-4663-8e62-d3b42942ee1f None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:36.424356 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905276c1-02fe-428e-b5d7-bbd61b98b8be service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a59e1d67ff94186a3ebcfcb49938a87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:36.425498 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905276c1-02fe-428e-b5d7-bbd61b98b8be service swift] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:36.426029 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905276c1-02fe-428e-b5d7-bbd61b98b8be service swift] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:36.426486 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905276c1-02fe-428e-b5d7-bbd61b98b8be service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:36.452925 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905276c1-02fe-428e-b5d7-bbd61b98b8be service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:36.454971 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905276c1-02fe-428e-b5d7-bbd61b98b8be service swift] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:36.476739 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-03b1dc50-4a68-495a-8aa6-da3a38ace17e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a254495b329146779d6841bb64c6af2d', 'request_id': 'req-03b1dc50-4a68-495a-8aa6-da3a38ace17e'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:36.477717 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1069/2146] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 2 19:22:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 02 19:22:36.559051 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-03b1dc50-4a68-495a-8aa6-da3a38ace17e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a254495b329146779d6841bb64c6af2d', 'request_id': 'req-03b1dc50-4a68-495a-8aa6-da3a38ace17e'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:36.565193 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-03b1dc50-4a68-495a-8aa6-da3a38ace17e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a254495b329146779d6841bb64c6af2d', 'request_id': 'req-03b1dc50-4a68-495a-8aa6-da3a38ace17e'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:36.577820 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-03b1dc50-4a68-495a-8aa6-da3a38ace17e admin admin] The token cache is being invalidate because project a254495b329146779d6841bb64c6af2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:22:36.578147 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-03b1dc50-4a68-495a-8aa6-da3a38ace17e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:36.578870 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-03b1dc50-4a68-495a-8aa6-da3a38ace17e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:36.580557 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1078/2147] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:22:36 2022] DELETE /identity/v3/projects/a254495b329146779d6841bb64c6af2d => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:22:36.586218 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd5fc8b-099d-4430-94ae-f670f7df82a1 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:36.586782 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd5fc8b-099d-4430-94ae-f670f7df82a1 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:36.587233 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd5fc8b-099d-4430-94ae-f670f7df82a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:36.618905 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-6cd5fc8b-099d-4430-94ae-f670f7df82a1 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:22:36.620105 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-6cd5fc8b-099d-4430-94ae-f670f7df82a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:36.699686 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1070/2148] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:22:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:22:36.707581 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dcdfb2-13a9-4469-a583-a0ec805cdf48 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:36.707907 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dcdfb2-13a9-4469-a583-a0ec805cdf48 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:36.708186 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dcdfb2-13a9-4469-a583-a0ec805cdf48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:36.716065 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:22:36.738774 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-81dcdfb2-13a9-4469-a583-a0ec805cdf48 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:22:36.738774 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-81dcdfb2-13a9-4469-a583-a0ec805cdf48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:36.744444 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1079/2149] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:22:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:22:36.748555 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd5fc8b-099d-4430-94ae-f670f7df82a1 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:36.759989 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd5fc8b-099d-4430-94ae-f670f7df82a1 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:36.765887 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e494647-ad9a-4944-8b0b-d69b1384200d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:36.767257 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e494647-ad9a-4944-8b0b-d69b1384200d admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:36.767569 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e494647-ad9a-4944-8b0b-d69b1384200d admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:36.768857 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e494647-ad9a-4944-8b0b-d69b1384200d admin admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:36.772131 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e494647-ad9a-4944-8b0b-d69b1384200d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:22:36.772516 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e494647-ad9a-4944-8b0b-d69b1384200d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:36.774310 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e494647-ad9a-4944-8b0b-d69b1384200d admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:36.789764 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1071/2150] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 2 19:22:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:22:36.794694 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dcdfb2-13a9-4469-a583-a0ec805cdf48 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:36.805434 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dcdfb2-13a9-4469-a583-a0ec805cdf48 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:36.816132 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615427fe-1966-4051-947a-b5ccadd69f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:36.817184 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615427fe-1966-4051-947a-b5ccadd69f69 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:36.817477 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615427fe-1966-4051-947a-b5ccadd69f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:36.817862 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615427fe-1966-4051-947a-b5ccadd69f69 admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:36.829195 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615427fe-1966-4051-947a-b5ccadd69f69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:36.831472 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615427fe-1966-4051-947a-b5ccadd69f69 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:36.980974 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1080/2151] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:22:36 2022] POST /identity/v3/projects => generated 377 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:22:36.983284 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e494647-ad9a-4944-8b0b-d69b1384200d admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:36.993964 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e494647-ad9a-4944-8b0b-d69b1384200d admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:37.096657 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26e534a-c934-4db1-b48b-28ad383f5e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:37.097539 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e534a-c934-4db1-b48b-28ad383f5e9d admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:37.098864 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e534a-c934-4db1-b48b-28ad383f5e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:37.098864 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e534a-c934-4db1-b48b-28ad383f5e9d admin admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:37.099179 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26e534a-c934-4db1-b48b-28ad383f5e9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:37.103863 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26e534a-c934-4db1-b48b-28ad383f5e9d admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:37.111771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:22:37.217298 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1072/2152] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:22:36 2022] POST /identity/v3/users => generated 289 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 02 19:22:37.225508 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615427fe-1966-4051-947a-b5ccadd69f69 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:37.238801 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615427fe-1966-4051-947a-b5ccadd69f69 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:37.245013 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db15296c-e71b-4d40-b5e0-6632a5a1de98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:37.246819 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db15296c-e71b-4d40-b5e0-6632a5a1de98 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:37.247087 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db15296c-e71b-4d40-b5e0-6632a5a1de98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:37.247369 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db15296c-e71b-4d40-b5e0-6632a5a1de98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:37.248338 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db15296c-e71b-4d40-b5e0-6632a5a1de98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:37.250943 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db15296c-e71b-4d40-b5e0-6632a5a1de98 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:37.263988 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1081/2153] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:22:37 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:22:37.268750 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26e534a-c934-4db1-b48b-28ad383f5e9d admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:37.288069 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26e534a-c934-4db1-b48b-28ad383f5e9d admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:37.293702 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5573895a-8759-405b-ba97-36a50edbb274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:37.294522 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5573895a-8759-405b-ba97-36a50edbb274 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:37.295226 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5573895a-8759-405b-ba97-36a50edbb274 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:37.295338 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5573895a-8759-405b-ba97-36a50edbb274 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:37.296314 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5573895a-8759-405b-ba97-36a50edbb274 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:37.298077 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5573895a-8759-405b-ba97-36a50edbb274 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:37.315882 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1073/2154] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:22:37 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:22:37.320868 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db15296c-e71b-4d40-b5e0-6632a5a1de98 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:37.332517 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db15296c-e71b-4d40-b5e0-6632a5a1de98 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:37.339552 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056c1682-1cd1-4f39-9eaa-18d03c92d921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:37.340512 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c1682-1cd1-4f39-9eaa-18d03c92d921 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:37.342822 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c1682-1cd1-4f39-9eaa-18d03c92d921 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:37.342822 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c1682-1cd1-4f39-9eaa-18d03c92d921 admin admin] PATH_INFO: `/v3/projects/883b3d816dd64abe83b8582530b03afa/users/33a5e782e5c84f7fbb6c21d72f092899/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:37.369291 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056c1682-1cd1-4f39-9eaa-18d03c92d921 admin admin] RBAC: Authorizing `identity:create_grant(project_id=883b3d816dd64abe83b8582530b03afa, user_id=33a5e782e5c84f7fbb6c21d72f092899, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:37.372100 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056c1682-1cd1-4f39-9eaa-18d03c92d921 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:37.469928 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1082/2155] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:22:37 2022] PUT /identity/v3/projects/883b3d816dd64abe83b8582530b03afa/users/33a5e782e5c84f7fbb6c21d72f092899/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:22:37.487445 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd9452-d334-499b-bf6b-1f26ad070cd7 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:37.488211 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd9452-d334-499b-bf6b-1f26ad070cd7 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:37.488211 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd9452-d334-499b-bf6b-1f26ad070cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:37.544876 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb8f8f-72fe-46ea-9885-6c98b1e9d698 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:37.545296 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb8f8f-72fe-46ea-9885-6c98b1e9d698 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:37.545654 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb8f8f-72fe-46ea-9885-6c98b1e9d698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:37.549488 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-15cd9452-d334-499b-bf6b-1f26ad070cd7 None None] MFA Rules not processed for user `33a5e782e5c84f7fbb6c21d72f092899`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:22:37.551108 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-15cd9452-d334-499b-bf6b-1f26ad070cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:37.571553 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-25cb8f8f-72fe-46ea-9885-6c98b1e9d698 None None] MFA Rules not processed for user `04f64813f09348eb91cb7bb2869f7612`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:22:37.574691 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-25cb8f8f-72fe-46ea-9885-6c98b1e9d698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:37.633426 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1074/2156] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:22:37 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:22:37.639317 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cd9452-d334-499b-bf6b-1f26ad070cd7 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:37.654395 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1083/2157] 10.222.0.33 () {58 vars in 967 bytes} [Thu Jun 2 19:22:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:22:37.671586 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cb8f8f-72fe-46ea-9885-6c98b1e9d698 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:37.673104 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-25cb8f8f-72fe-46ea-9885-6c98b1e9d698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:37.750905 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cd9452-d334-499b-bf6b-1f26ad070cd7 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:37.756533 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f86415f-190a-40db-a815-b72aff3d5d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:37.757208 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f86415f-190a-40db-a815-b72aff3d5d08 admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:37.757494 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f86415f-190a-40db-a815-b72aff3d5d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:37.757773 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f86415f-190a-40db-a815-b72aff3d5d08 admin admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:37.758486 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f86415f-190a-40db-a815-b72aff3d5d08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:37.760803 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f86415f-190a-40db-a815-b72aff3d5d08 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:37.766487 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cb8f8f-72fe-46ea-9885-6c98b1e9d698 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:37.771554 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516132d1-c84c-4bf8-b408-83207a299b66 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a59e1d67ff94186a3ebcfcb49938a87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:37.772282 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516132d1-c84c-4bf8-b408-83207a299b66 service swift] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:37.772629 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516132d1-c84c-4bf8-b408-83207a299b66 service swift] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:37.772978 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516132d1-c84c-4bf8-b408-83207a299b66 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:37.783363 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516132d1-c84c-4bf8-b408-83207a299b66 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:37.785081 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516132d1-c84c-4bf8-b408-83207a299b66 service swift] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:37.795409 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1084/2158] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 2 19:22:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 02 19:22:37.837527 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1075/2159] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:22:37 2022] POST /identity/v3/projects => generated 379 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 02 19:22:37.842149 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516132d1-c84c-4bf8-b408-83207a299b66 service swift] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:37.853748 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516132d1-c84c-4bf8-b408-83207a299b66 service swift] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:37.884156 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:22:37.915528 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9118f69-81c6-4af0-8aac-45b4e8e0e313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:37.915528 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9118f69-81c6-4af0-8aac-45b4e8e0e313 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:37.915528 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9118f69-81c6-4af0-8aac-45b4e8e0e313 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:37.915528 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9118f69-81c6-4af0-8aac-45b4e8e0e313 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:37.916821 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9118f69-81c6-4af0-8aac-45b4e8e0e313 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:37.918871 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9118f69-81c6-4af0-8aac-45b4e8e0e313 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:38.035067 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1085/2160] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:22:37 2022] POST /identity/v3/users => generated 290 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:22:38.039907 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f86415f-190a-40db-a815-b72aff3d5d08 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:38.051419 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f86415f-190a-40db-a815-b72aff3d5d08 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:38.056525 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af727835-54db-43d5-8fee-500ff1183d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:38.057238 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af727835-54db-43d5-8fee-500ff1183d38 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:38.057521 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af727835-54db-43d5-8fee-500ff1183d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:38.057797 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af727835-54db-43d5-8fee-500ff1183d38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:38.058831 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af727835-54db-43d5-8fee-500ff1183d38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:38.060471 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af727835-54db-43d5-8fee-500ff1183d38 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:38.071858 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1076/2161] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:22:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 02 19:22:38.076401 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9118f69-81c6-4af0-8aac-45b4e8e0e313 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:38.090965 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9118f69-81c6-4af0-8aac-45b4e8e0e313 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:38.096376 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9e4f40-779b-4568-a1e9-0e846d853215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:38.097207 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e4f40-779b-4568-a1e9-0e846d853215 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:38.097330 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e4f40-779b-4568-a1e9-0e846d853215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:38.097655 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e4f40-779b-4568-a1e9-0e846d853215 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:38.098560 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9e4f40-779b-4568-a1e9-0e846d853215 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:38.100916 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9e4f40-779b-4568-a1e9-0e846d853215 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:38.114979 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1086/2162] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:22:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:22:38.116271 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af727835-54db-43d5-8fee-500ff1183d38 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:38.127653 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af727835-54db-43d5-8fee-500ff1183d38 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:38.132374 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fe0529-073d-42c5-bac6-94bbc0838415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:38.133157 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fe0529-073d-42c5-bac6-94bbc0838415 admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:38.133473 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fe0529-073d-42c5-bac6-94bbc0838415 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:38.133751 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fe0529-073d-42c5-bac6-94bbc0838415 admin admin] PATH_INFO: `/v3/projects/2a5f5458e52a43e78494327f7ed78f19/users/dd568e9fd7304ac4ab13c0f3d7b4543f/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:38.150833 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fe0529-073d-42c5-bac6-94bbc0838415 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a5f5458e52a43e78494327f7ed78f19, user_id=dd568e9fd7304ac4ab13c0f3d7b4543f, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:38.154045 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fe0529-073d-42c5-bac6-94bbc0838415 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:38.240789 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:22:38.246972 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1077/2163] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:22:38 2022] PUT /identity/v3/projects/2a5f5458e52a43e78494327f7ed78f19/users/dd568e9fd7304ac4ab13c0f3d7b4543f/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:22:38.258980 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb0f95-ebff-4fbb-9747-12cc50e2b8b9 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:38.259507 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb0f95-ebff-4fbb-9747-12cc50e2b8b9 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:38.259600 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb0f95-ebff-4fbb-9747-12cc50e2b8b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:38.304290 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-33bb0f95-ebff-4fbb-9747-12cc50e2b8b9 None None] MFA Rules not processed for user `dd568e9fd7304ac4ab13c0f3d7b4543f`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:22:38.306083 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-33bb0f95-ebff-4fbb-9747-12cc50e2b8b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:38.382616 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1087/2164] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:22:38 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:22:38.386987 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fe0529-073d-42c5-bac6-94bbc0838415 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:38.470861 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7661a806-bce9-485f-9172-13e14bd81ff1 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:38.471566 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7661a806-bce9-485f-9172-13e14bd81ff1 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:38.472006 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fe0529-073d-42c5-bac6-94bbc0838415 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:38.472349 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7661a806-bce9-485f-9172-13e14bd81ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:38.476839 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a7e2da-02bf-4cd9-9300-a8d080eb8504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:38.477729 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a7e2da-02bf-4cd9-9300-a8d080eb8504 admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:38.478025 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a7e2da-02bf-4cd9-9300-a8d080eb8504 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:38.478302 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a7e2da-02bf-4cd9-9300-a8d080eb8504 admin admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:38.479051 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a7e2da-02bf-4cd9-9300-a8d080eb8504 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:38.482021 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a7e2da-02bf-4cd9-9300-a8d080eb8504 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:38.497755 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-7661a806-bce9-485f-9172-13e14bd81ff1 None None] MFA Rules not processed for user `04f64813f09348eb91cb7bb2869f7612`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:22:38.499016 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-7661a806-bce9-485f-9172-13e14bd81ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:38.563657 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1088/2165] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Jun 2 19:22:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:22:38.576840 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7661a806-bce9-485f-9172-13e14bd81ff1 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:38.593879 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1078/2166] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:22:38 2022] POST /identity/v3/projects => generated 377 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:22:38.598170 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a7e2da-02bf-4cd9-9300-a8d080eb8504 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:38.655105 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:22:38.677151 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7661a806-bce9-485f-9172-13e14bd81ff1 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:38.702259 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a7e2da-02bf-4cd9-9300-a8d080eb8504 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:38.706989 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb20bb5-94ee-454d-8c86-c365b8140b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:38.708363 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb20bb5-94ee-454d-8c86-c365b8140b4b admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:38.708658 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb20bb5-94ee-454d-8c86-c365b8140b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:38.708979 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb20bb5-94ee-454d-8c86-c365b8140b4b admin admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:38.709716 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb20bb5-94ee-454d-8c86-c365b8140b4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:38.711775 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb20bb5-94ee-454d-8c86-c365b8140b4b admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:38.742612 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a5ced5-dd9c-4e0a-ab19-6b88fb921eef service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a59e1d67ff94186a3ebcfcb49938a87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:38.743557 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a5ced5-dd9c-4e0a-ab19-6b88fb921eef service swift] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:38.743557 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a5ced5-dd9c-4e0a-ab19-6b88fb921eef service swift] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:38.743849 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a5ced5-dd9c-4e0a-ab19-6b88fb921eef service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:38.805529 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a5ced5-dd9c-4e0a-ab19-6b88fb921eef service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:38.807232 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a5ced5-dd9c-4e0a-ab19-6b88fb921eef service swift] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:38.816291 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1089/2167] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 2 19:22:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 02 19:22:38.821263 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1079/2168] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:22:38 2022] POST /identity/v3/users => generated 295 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:22:38.825222 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a5ced5-dd9c-4e0a-ab19-6b88fb921eef service swift] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:38.836839 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a5ced5-dd9c-4e0a-ab19-6b88fb921eef service swift] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:38.843746 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8267325c-9c14-483e-a93e-07e4f200e764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:38.848495 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8267325c-9c14-483e-a93e-07e4f200e764 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:38.848495 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8267325c-9c14-483e-a93e-07e4f200e764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:38.848495 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8267325c-9c14-483e-a93e-07e4f200e764 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:38.848495 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8267325c-9c14-483e-a93e-07e4f200e764 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:38.849324 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8267325c-9c14-483e-a93e-07e4f200e764 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:38.860658 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1090/2169] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:22:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:22:38.866582 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb20bb5-94ee-454d-8c86-c365b8140b4b admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:38.886411 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb20bb5-94ee-454d-8c86-c365b8140b4b admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:38.894595 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46a8311-f8a7-4566-be3a-7226ca8b5ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:38.896246 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a8311-f8a7-4566-be3a-7226ca8b5ea7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:38.896768 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a8311-f8a7-4566-be3a-7226ca8b5ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:38.897298 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a8311-f8a7-4566-be3a-7226ca8b5ea7 admin admin] PATH_INFO: `/v3/projects/eea283c6fd8940f3be3c840217f0baa0/users/e2cb2cdf40fe4a6f94a7168d19754ff4/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:38.926323 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46a8311-f8a7-4566-be3a-7226ca8b5ea7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eea283c6fd8940f3be3c840217f0baa0, user_id=e2cb2cdf40fe4a6f94a7168d19754ff4, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:38.932421 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46a8311-f8a7-4566-be3a-7226ca8b5ea7 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:39.059767 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1080/2170] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:22:38 2022] PUT /identity/v3/projects/eea283c6fd8940f3be3c840217f0baa0/users/e2cb2cdf40fe4a6f94a7168d19754ff4/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:22:39.064000 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8267325c-9c14-483e-a93e-07e4f200e764 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:39.074580 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8267325c-9c14-483e-a93e-07e4f200e764 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:39.141035 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec96b81-3c76-48f0-8b61-b26c4dc2ca57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:39.141914 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec96b81-3c76-48f0-8b61-b26c4dc2ca57 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:39.142533 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec96b81-3c76-48f0-8b61-b26c4dc2ca57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:39.143176 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec96b81-3c76-48f0-8b61-b26c4dc2ca57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:39.144051 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec96b81-3c76-48f0-8b61-b26c4dc2ca57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:39.146216 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec96b81-3c76-48f0-8b61-b26c4dc2ca57 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:39.158774 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1091/2171] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:22:39 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:22:39.164599 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46a8311-f8a7-4566-be3a-7226ca8b5ea7 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:39.173987 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46a8311-f8a7-4566-be3a-7226ca8b5ea7 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:39.179348 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbb1250-8385-4745-bd04-97037ad40131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:39.180230 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbb1250-8385-4745-bd04-97037ad40131 admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:39.180612 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbb1250-8385-4745-bd04-97037ad40131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:39.180841 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbb1250-8385-4745-bd04-97037ad40131 admin admin] PATH_INFO: `/v3/projects/eea283c6fd8940f3be3c840217f0baa0/users/e2cb2cdf40fe4a6f94a7168d19754ff4/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:39.182854 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbb1250-8385-4745-bd04-97037ad40131 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eea283c6fd8940f3be3c840217f0baa0, user_id=e2cb2cdf40fe4a6f94a7168d19754ff4, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:39.185480 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbb1250-8385-4745-bd04-97037ad40131 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:39.196861 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1081/2172] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:22:39 2022] PUT /identity/v3/projects/eea283c6fd8940f3be3c840217f0baa0/users/e2cb2cdf40fe4a6f94a7168d19754ff4/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:22:39.207048 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd8cc8e-6898-4df1-8316-95b6f4b9ea45 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:39.207336 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd8cc8e-6898-4df1-8316-95b6f4b9ea45 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:39.207614 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd8cc8e-6898-4df1-8316-95b6f4b9ea45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:39.236123 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-0bd8cc8e-6898-4df1-8316-95b6f4b9ea45 None None] MFA Rules not processed for user `e2cb2cdf40fe4a6f94a7168d19754ff4`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:22:39.237414 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-0bd8cc8e-6898-4df1-8316-95b6f4b9ea45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:39.242001 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:22:39.311528 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1092/2173] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:22:39 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:22:39.323210 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbb1250-8385-4745-bd04-97037ad40131 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:39.324916 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-ccbb1250-8385-4745-bd04-97037ad40131 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:39.377047 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:22:39.395012 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbb1250-8385-4745-bd04-97037ad40131 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:39.399754 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e07368a-1db0-4d9e-a6a6-e507e8271e93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:39.400281 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07368a-1db0-4d9e-a6a6-e507e8271e93 service cinder] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:39.400571 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07368a-1db0-4d9e-a6a6-e507e8271e93 service cinder] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:39.400846 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07368a-1db0-4d9e-a6a6-e507e8271e93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:39.411384 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e07368a-1db0-4d9e-a6a6-e507e8271e93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:39.413526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e07368a-1db0-4d9e-a6a6-e507e8271e93 service cinder] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:39.424832 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1082/2174] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:22:39 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:22:39.534078 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb8127b-eddd-47d8-bcbb-245b60256275 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:39.534078 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb8127b-eddd-47d8-bcbb-245b60256275 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:39.534078 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb8127b-eddd-47d8-bcbb-245b60256275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:39.553924 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-7fb8127b-eddd-47d8-bcbb-245b60256275 None None] MFA Rules not processed for user `04f64813f09348eb91cb7bb2869f7612`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:22:39.555225 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-7fb8127b-eddd-47d8-bcbb-245b60256275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:39.555365 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e07368a-1db0-4d9e-a6a6-e507e8271e93 service cinder] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:39.556966 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-0e07368a-1db0-4d9e-a6a6-e507e8271e93 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:39.569985 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e07368a-1db0-4d9e-a6a6-e507e8271e93 service cinder] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:39.575952 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d269060-1e15-4ecc-8fb3-b4553889c84c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:39.576901 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d269060-1e15-4ecc-8fb3-b4553889c84c service cinder] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:39.577344 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d269060-1e15-4ecc-8fb3-b4553889c84c service cinder] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:39.577778 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d269060-1e15-4ecc-8fb3-b4553889c84c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:39.631825 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1093/2175] 10.222.0.33 () {58 vars in 967 bytes} [Thu Jun 2 19:22:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:22:39.646972 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb8127b-eddd-47d8-bcbb-245b60256275 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:39.695925 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d269060-1e15-4ecc-8fb3-b4553889c84c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:39.701346 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d269060-1e15-4ecc-8fb3-b4553889c84c service cinder] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:39.730315 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1083/2176] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:22:39 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 02 19:22:39.774359 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb8127b-eddd-47d8-bcbb-245b60256275 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:39.780700 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17816c9c-16bc-424a-b4cb-f1a3ee2b0b9c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a59e1d67ff94186a3ebcfcb49938a87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:39.781366 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17816c9c-16bc-424a-b4cb-f1a3ee2b0b9c service swift] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:39.781657 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17816c9c-16bc-424a-b4cb-f1a3ee2b0b9c service swift] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:39.781946 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17816c9c-16bc-424a-b4cb-f1a3ee2b0b9c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:39.800560 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17816c9c-16bc-424a-b4cb-f1a3ee2b0b9c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:39.803606 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17816c9c-16bc-424a-b4cb-f1a3ee2b0b9c service swift] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:39.819417 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1094/2177] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 2 19:22:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 02 19:22:42.901195 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d269060-1e15-4ecc-8fb3-b4553889c84c service cinder] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:42.905556 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-8d269060-1e15-4ecc-8fb3-b4553889c84c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:43.006888 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d269060-1e15-4ecc-8fb3-b4553889c84c service cinder] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:43.012485 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62973307-9e92-42fc-bdcb-6cc0fddff8a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:43.013732 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62973307-9e92-42fc-bdcb-6cc0fddff8a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:43.014026 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62973307-9e92-42fc-bdcb-6cc0fddff8a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:43.014294 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62973307-9e92-42fc-bdcb-6cc0fddff8a1 admin admin] PATH_INFO: `/v3/users/d38ac88237f54a0895a8c04e2a9290a9` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:43.016355 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62973307-9e92-42fc-bdcb-6cc0fddff8a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d38ac88237f54a0895a8c04e2a9290a9)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:43.018793 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62973307-9e92-42fc-bdcb-6cc0fddff8a1 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:43.278302 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-62973307-9e92-42fc-bdcb-6cc0fddff8a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd38ac88237f54a0895a8c04e2a9290a9'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:43.286730 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-62973307-9e92-42fc-bdcb-6cc0fddff8a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd38ac88237f54a0895a8c04e2a9290a9'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:43.287571 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-62973307-9e92-42fc-bdcb-6cc0fddff8a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd38ac88237f54a0895a8c04e2a9290a9'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:43.360031 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-62973307-9e92-42fc-bdcb-6cc0fddff8a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd38ac88237f54a0895a8c04e2a9290a9'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:43.360776 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-62973307-9e92-42fc-bdcb-6cc0fddff8a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd38ac88237f54a0895a8c04e2a9290a9'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:43.379611 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:22:43.386601 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1084/2178] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:22:42 2022] DELETE /identity/v3/users/d38ac88237f54a0895a8c04e2a9290a9 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 02 19:22:43.391320 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c9b1aa-c30d-4fa1-85e4-c378f13907c3 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:43.392038 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c9b1aa-c30d-4fa1-85e4-c378f13907c3 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:43.392038 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c9b1aa-c30d-4fa1-85e4-c378f13907c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:43.417730 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-b4c9b1aa-c30d-4fa1-85e4-c378f13907c3 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:22:43.419994 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-b4c9b1aa-c30d-4fa1-85e4-c378f13907c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:43.459814 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:22:43.502385 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1095/2179] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:22:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 19:22:43.513454 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62973307-9e92-42fc-bdcb-6cc0fddff8a1 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:43.515727 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-62973307-9e92-42fc-bdcb-6cc0fddff8a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:43.602967 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62973307-9e92-42fc-bdcb-6cc0fddff8a1 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:43.606854 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c75362d-e93d-42dc-bca2-92af703ad32c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:43.607525 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75362d-e93d-42dc-bca2-92af703ad32c service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:43.623249 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75362d-e93d-42dc-bca2-92af703ad32c service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:43.623249 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75362d-e93d-42dc-bca2-92af703ad32c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:43.623249 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c75362d-e93d-42dc-bca2-92af703ad32c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:43.623249 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c75362d-e93d-42dc-bca2-92af703ad32c service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:43.632134 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1085/2180] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:22:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:22:44.010207 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c9b1aa-c30d-4fa1-85e4-c378f13907c3 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:44.013876 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-b4c9b1aa-c30d-4fa1-85e4-c378f13907c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:44.044512 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c9b1aa-c30d-4fa1-85e4-c378f13907c3 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:44.056224 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b4d549-464f-4dac-854b-a7d1b3bff868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:44.058311 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b4d549-464f-4dac-854b-a7d1b3bff868 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:44.059122 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b4d549-464f-4dac-854b-a7d1b3bff868 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:44.059916 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b4d549-464f-4dac-854b-a7d1b3bff868 admin admin] PATH_INFO: `/v3/projects/1e2cdcd22e66456da3a49a5333f64de2` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:44.062426 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b4d549-464f-4dac-854b-a7d1b3bff868 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e2cdcd22e66456da3a49a5333f64de2)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:44.067004 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b4d549-464f-4dac-854b-a7d1b3bff868 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:44.411058 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-d6b4d549-464f-4dac-854b-a7d1b3bff868 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e2cdcd22e66456da3a49a5333f64de2', 'request_id': 'req-d6b4d549-464f-4dac-854b-a7d1b3bff868'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:44.606376 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-d6b4d549-464f-4dac-854b-a7d1b3bff868 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e2cdcd22e66456da3a49a5333f64de2', 'request_id': 'req-d6b4d549-464f-4dac-854b-a7d1b3bff868'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:44.620286 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-d6b4d549-464f-4dac-854b-a7d1b3bff868 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e2cdcd22e66456da3a49a5333f64de2', 'request_id': 'req-d6b4d549-464f-4dac-854b-a7d1b3bff868'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:44.650084 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c75362d-e93d-42dc-bca2-92af703ad32c service neutron] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:44.656954 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-9c75362d-e93d-42dc-bca2-92af703ad32c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:44.657171 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-d6b4d549-464f-4dac-854b-a7d1b3bff868 admin admin] The token cache is being invalidate because project 1e2cdcd22e66456da3a49a5333f64de2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:22:44.657171 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-d6b4d549-464f-4dac-854b-a7d1b3bff868 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:44.657171 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-d6b4d549-464f-4dac-854b-a7d1b3bff868 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:44.657583 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1096/2181] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:22:44 2022] DELETE /identity/v3/projects/1e2cdcd22e66456da3a49a5333f64de2 => generated 0 bytes in 649 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:22:44.665100 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9605a0-146d-48d0-8b0c-80a70cb618ec None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:44.665472 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9605a0-146d-48d0-8b0c-80a70cb618ec None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:44.665960 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9605a0-146d-48d0-8b0c-80a70cb618ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:44.681652 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c75362d-e93d-42dc-bca2-92af703ad32c service neutron] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:44.741966 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-1d9605a0-146d-48d0-8b0c-80a70cb618ec None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:22:44.748309 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-1d9605a0-146d-48d0-8b0c-80a70cb618ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:44.796009 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508d380d-d7e8-4ee2-837f-098baba4b50b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:44.800592 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508d380d-d7e8-4ee2-837f-098baba4b50b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:44.800592 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508d380d-d7e8-4ee2-837f-098baba4b50b admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:44.800592 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508d380d-d7e8-4ee2-837f-098baba4b50b admin admin] PATH_INFO: `/v3/users/33a5e782e5c84f7fbb6c21d72f092899` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:44.800592 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508d380d-d7e8-4ee2-837f-098baba4b50b admin admin] RBAC: Authorizing `identity:delete_user(user_id=33a5e782e5c84f7fbb6c21d72f092899)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:44.805554 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508d380d-d7e8-4ee2-837f-098baba4b50b admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:44.896745 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1097/2182] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:22:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 19:22:44.909105 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc5a3ca-682b-47ad-85a1-14881250dc41 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:44.912880 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc5a3ca-682b-47ad-85a1-14881250dc41 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:44.912880 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc5a3ca-682b-47ad-85a1-14881250dc41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:44.942187 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-8cc5a3ca-682b-47ad-85a1-14881250dc41 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:22:44.944377 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-8cc5a3ca-682b-47ad-85a1-14881250dc41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:44.954119 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1098/2183] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:22:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 19:22:44.959115 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc5a3ca-682b-47ad-85a1-14881250dc41 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:44.969699 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc5a3ca-682b-47ad-85a1-14881250dc41 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:44.975346 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a1c1f9-5ee5-467e-b670-323d62743ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:44.981675 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a1c1f9-5ee5-467e-b670-323d62743ee1 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:44.981675 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a1c1f9-5ee5-467e-b670-323d62743ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:44.981675 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a1c1f9-5ee5-467e-b670-323d62743ee1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:44.981675 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a1c1f9-5ee5-467e-b670-323d62743ee1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:22:44.981675 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a1c1f9-5ee5-467e-b670-323d62743ee1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:44.981675 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a1c1f9-5ee5-467e-b670-323d62743ee1 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:44.993590 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1099/2184] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 2 19:22:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:22:44.998051 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a1c1f9-5ee5-467e-b670-323d62743ee1 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:45.008500 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a1c1f9-5ee5-467e-b670-323d62743ee1 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:45.013606 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d0429e-1f9f-4a93-9d30-e6df3a9aad69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:45.016310 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d0429e-1f9f-4a93-9d30-e6df3a9aad69 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:45.016310 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d0429e-1f9f-4a93-9d30-e6df3a9aad69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:45.016310 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d0429e-1f9f-4a93-9d30-e6df3a9aad69 admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:45.016310 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d0429e-1f9f-4a93-9d30-e6df3a9aad69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:45.017653 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d0429e-1f9f-4a93-9d30-e6df3a9aad69 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:45.091850 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-508d380d-d7e8-4ee2-837f-098baba4b50b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33a5e782e5c84f7fbb6c21d72f092899'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:45.100821 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-508d380d-d7e8-4ee2-837f-098baba4b50b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33a5e782e5c84f7fbb6c21d72f092899'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:45.101763 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-508d380d-d7e8-4ee2-837f-098baba4b50b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33a5e782e5c84f7fbb6c21d72f092899'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:45.107633 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1100/2185] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:22:44 2022] POST /identity/v3/projects => generated 383 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 02 19:22:45.113775 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d0429e-1f9f-4a93-9d30-e6df3a9aad69 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:45.116608 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-44d0429e-1f9f-4a93-9d30-e6df3a9aad69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:45.133169 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:22:45.220406 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-508d380d-d7e8-4ee2-837f-098baba4b50b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33a5e782e5c84f7fbb6c21d72f092899'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:45.221661 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-508d380d-d7e8-4ee2-837f-098baba4b50b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33a5e782e5c84f7fbb6c21d72f092899'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:45.227178 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d0429e-1f9f-4a93-9d30-e6df3a9aad69 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:45.235702 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b8582e-77b9-4b54-9fe2-bd2af60b23dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:45.237249 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b8582e-77b9-4b54-9fe2-bd2af60b23dd admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:45.237854 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b8582e-77b9-4b54-9fe2-bd2af60b23dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:45.238331 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b8582e-77b9-4b54-9fe2-bd2af60b23dd admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:45.239606 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b8582e-77b9-4b54-9fe2-bd2af60b23dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:45.243083 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b8582e-77b9-4b54-9fe2-bd2af60b23dd admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:45.252829 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1086/2186] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:22:44 2022] DELETE /identity/v3/users/33a5e782e5c84f7fbb6c21d72f092899 => generated 0 bytes in 603 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:22:45.259724 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d20ae1-0281-41d7-b717-9e39539e8da5 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:45.260023 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d20ae1-0281-41d7-b717-9e39539e8da5 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:45.260300 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d20ae1-0281-41d7-b717-9e39539e8da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:45.280834 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-a2d20ae1-0281-41d7-b717-9e39539e8da5 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:22:45.283361 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-a2d20ae1-0281-41d7-b717-9e39539e8da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:45.296548 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:22:45.360330 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1087/2187] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:22:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 19:22:45.375884 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d20ae1-0281-41d7-b717-9e39539e8da5 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:45.377435 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1101/2188] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:22:45 2022] POST /identity/v3/users => generated 292 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 02 19:22:45.379818 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-a2d20ae1-0281-41d7-b717-9e39539e8da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:45.383822 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b8582e-77b9-4b54-9fe2-bd2af60b23dd admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:45.403690 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b8582e-77b9-4b54-9fe2-bd2af60b23dd admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:45.412254 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63db2404-11a4-4f6b-ba5c-ef131f19f86a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:45.413610 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63db2404-11a4-4f6b-ba5c-ef131f19f86a admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:45.414236 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63db2404-11a4-4f6b-ba5c-ef131f19f86a admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:45.415209 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63db2404-11a4-4f6b-ba5c-ef131f19f86a admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:45.416904 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63db2404-11a4-4f6b-ba5c-ef131f19f86a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:45.420160 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63db2404-11a4-4f6b-ba5c-ef131f19f86a admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:45.444736 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1102/2189] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:22:45 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 02 19:22:45.446256 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63db2404-11a4-4f6b-ba5c-ef131f19f86a admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:45.461880 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63db2404-11a4-4f6b-ba5c-ef131f19f86a admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:45.467852 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af1a2b9-eeab-4c05-908b-37e2db4543d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:45.468735 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af1a2b9-eeab-4c05-908b-37e2db4543d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:45.469120 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af1a2b9-eeab-4c05-908b-37e2db4543d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:45.469579 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af1a2b9-eeab-4c05-908b-37e2db4543d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:45.469747 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d20ae1-0281-41d7-b717-9e39539e8da5 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:45.470626 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af1a2b9-eeab-4c05-908b-37e2db4543d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:45.472864 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af1a2b9-eeab-4c05-908b-37e2db4543d0 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:45.474500 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f526f7c-68f6-4bd5-a132-8030a1740605 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:45.475480 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f526f7c-68f6-4bd5-a132-8030a1740605 service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:45.476047 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f526f7c-68f6-4bd5-a132-8030a1740605 service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:45.476486 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f526f7c-68f6-4bd5-a132-8030a1740605 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:45.490842 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f526f7c-68f6-4bd5-a132-8030a1740605 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:45.491442 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1103/2190] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:22:45 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 02 19:22:45.494224 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f526f7c-68f6-4bd5-a132-8030a1740605 service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:45.498959 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af1a2b9-eeab-4c05-908b-37e2db4543d0 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:45.511365 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1088/2191] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:22:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:22:45.519113 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af1a2b9-eeab-4c05-908b-37e2db4543d0 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:45.528284 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2258166-e70f-4bad-a52e-7a51f615059f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:45.529936 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2258166-e70f-4bad-a52e-7a51f615059f admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:45.530443 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2258166-e70f-4bad-a52e-7a51f615059f admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:45.530979 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2258166-e70f-4bad-a52e-7a51f615059f admin admin] PATH_INFO: `/v3/projects/ccc7e404c01743078a73e9e3f79c0f22/users/e98d8cc46eb84774bc0ddaf8f1b17690/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:45.554565 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2258166-e70f-4bad-a52e-7a51f615059f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccc7e404c01743078a73e9e3f79c0f22, user_id=e98d8cc46eb84774bc0ddaf8f1b17690, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:45.557877 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2258166-e70f-4bad-a52e-7a51f615059f admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:45.647881 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1104/2192] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:22:45 2022] PUT /identity/v3/projects/ccc7e404c01743078a73e9e3f79c0f22/users/e98d8cc46eb84774bc0ddaf8f1b17690/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:22:45.653549 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0bf8de-d7c9-4331-b46c-d29e20185a74 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:45.654341 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0bf8de-d7c9-4331-b46c-d29e20185a74 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:45.654541 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0bf8de-d7c9-4331-b46c-d29e20185a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:45.674961 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-1a0bf8de-d7c9-4331-b46c-d29e20185a74 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:22:45.676354 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-1a0bf8de-d7c9-4331-b46c-d29e20185a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:45.768266 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1089/2193] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:22:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 19:22:45.791713 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2258166-e70f-4bad-a52e-7a51f615059f admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:45.794658 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-a2258166-e70f-4bad-a52e-7a51f615059f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:45.892207 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2258166-e70f-4bad-a52e-7a51f615059f admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:45.898137 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6d8b9e-97d8-40f8-995a-0c8597361790 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:45.899060 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6d8b9e-97d8-40f8-995a-0c8597361790 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:45.899387 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6d8b9e-97d8-40f8-995a-0c8597361790 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:45.899709 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6d8b9e-97d8-40f8-995a-0c8597361790 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:45.949299 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6d8b9e-97d8-40f8-995a-0c8597361790 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:45.972382 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6d8b9e-97d8-40f8-995a-0c8597361790 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:45.972382 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1105/2194] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:22:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:22:46.470899 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0bf8de-d7c9-4331-b46c-d29e20185a74 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:46.473277 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-1a0bf8de-d7c9-4331-b46c-d29e20185a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:46.525379 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0bf8de-d7c9-4331-b46c-d29e20185a74 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:46.540240 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d6cdb-2aa0-432e-8aea-28795236ba15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:46.546036 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d6cdb-2aa0-432e-8aea-28795236ba15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:46.546036 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d6cdb-2aa0-432e-8aea-28795236ba15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:46.546036 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d6cdb-2aa0-432e-8aea-28795236ba15 admin admin] PATH_INFO: `/v3/projects/883b3d816dd64abe83b8582530b03afa` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:46.546036 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8d6cdb-2aa0-432e-8aea-28795236ba15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=883b3d816dd64abe83b8582530b03afa)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:46.546036 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8d6cdb-2aa0-432e-8aea-28795236ba15 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:46.702004 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-4a8d6cdb-2aa0-432e-8aea-28795236ba15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '883b3d816dd64abe83b8582530b03afa', 'request_id': 'req-4a8d6cdb-2aa0-432e-8aea-28795236ba15'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:46.765568 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-4a8d6cdb-2aa0-432e-8aea-28795236ba15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '883b3d816dd64abe83b8582530b03afa', 'request_id': 'req-4a8d6cdb-2aa0-432e-8aea-28795236ba15'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:46.769002 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:22:46.776742 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-4a8d6cdb-2aa0-432e-8aea-28795236ba15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '883b3d816dd64abe83b8582530b03afa', 'request_id': 'req-4a8d6cdb-2aa0-432e-8aea-28795236ba15'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:46.796057 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-4a8d6cdb-2aa0-432e-8aea-28795236ba15 admin admin] The token cache is being invalidate because project 883b3d816dd64abe83b8582530b03afa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:22:46.796347 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-4a8d6cdb-2aa0-432e-8aea-28795236ba15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:46.797013 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-4a8d6cdb-2aa0-432e-8aea-28795236ba15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:46.802403 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1090/2195] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:22:46 2022] DELETE /identity/v3/projects/883b3d816dd64abe83b8582530b03afa => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:22:46.805497 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6d8b9e-97d8-40f8-995a-0c8597361790 service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:46.825177 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6d8b9e-97d8-40f8-995a-0c8597361790 service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:46.896097 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:22:46.905300 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a439cb-693f-42db-a4f4-81a00e637278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:46.906218 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a439cb-693f-42db-a4f4-81a00e637278 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:46.906678 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a439cb-693f-42db-a4f4-81a00e637278 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:46.907162 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a439cb-693f-42db-a4f4-81a00e637278 admin admin] PATH_INFO: `/v3/users/dd568e9fd7304ac4ab13c0f3d7b4543f` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:46.908478 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a439cb-693f-42db-a4f4-81a00e637278 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd568e9fd7304ac4ab13c0f3d7b4543f)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:46.911169 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a439cb-693f-42db-a4f4-81a00e637278 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:47.122941 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-b3a439cb-693f-42db-a4f4-81a00e637278 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd568e9fd7304ac4ab13c0f3d7b4543f'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:47.130461 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-b3a439cb-693f-42db-a4f4-81a00e637278 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd568e9fd7304ac4ab13c0f3d7b4543f'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:47.222473 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-b3a439cb-693f-42db-a4f4-81a00e637278 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd568e9fd7304ac4ab13c0f3d7b4543f'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:47.223168 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-b3a439cb-693f-42db-a4f4-81a00e637278 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd568e9fd7304ac4ab13c0f3d7b4543f'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:47.223852 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-b3a439cb-693f-42db-a4f4-81a00e637278 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd568e9fd7304ac4ab13c0f3d7b4543f'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:47.267440 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1106/2196] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:22:46 2022] DELETE /identity/v3/users/dd568e9fd7304ac4ab13c0f3d7b4543f => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:22:47.913165 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d6cdb-2aa0-432e-8aea-28795236ba15 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:47.914704 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-4a8d6cdb-2aa0-432e-8aea-28795236ba15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:47.952278 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d6cdb-2aa0-432e-8aea-28795236ba15 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:48.055050 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d793fb-c7b9-43c7-9afa-39007b411ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:48.056244 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d793fb-c7b9-43c7-9afa-39007b411ab8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:48.056633 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d793fb-c7b9-43c7-9afa-39007b411ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:48.056997 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d793fb-c7b9-43c7-9afa-39007b411ab8 admin admin] PATH_INFO: `/v3/projects/2a5f5458e52a43e78494327f7ed78f19` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:48.058489 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d793fb-c7b9-43c7-9afa-39007b411ab8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a5f5458e52a43e78494327f7ed78f19)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:48.061024 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d793fb-c7b9-43c7-9afa-39007b411ab8 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:48.115587 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f828393-b861-4eb8-9582-bc5cac533ebc None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:48.116540 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f828393-b861-4eb8-9582-bc5cac533ebc None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:48.117011 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f828393-b861-4eb8-9582-bc5cac533ebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:48.139752 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-9f828393-b861-4eb8-9582-bc5cac533ebc None None] MFA Rules not processed for user `7f950b5cc5fa44c988b945e9cd553874`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:22:48.141343 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-9f828393-b861-4eb8-9582-bc5cac533ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:48.224789 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1107/2197] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:22:48 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:22:48.247898 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-a1d793fb-c7b9-43c7-9afa-39007b411ab8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a5f5458e52a43e78494327f7ed78f19', 'request_id': 'req-a1d793fb-c7b9-43c7-9afa-39007b411ab8'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:48.342302 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-a1d793fb-c7b9-43c7-9afa-39007b411ab8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a5f5458e52a43e78494327f7ed78f19', 'request_id': 'req-a1d793fb-c7b9-43c7-9afa-39007b411ab8'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:48.351684 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-a1d793fb-c7b9-43c7-9afa-39007b411ab8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a5f5458e52a43e78494327f7ed78f19', 'request_id': 'req-a1d793fb-c7b9-43c7-9afa-39007b411ab8'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:48.392225 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-a1d793fb-c7b9-43c7-9afa-39007b411ab8 admin admin] The token cache is being invalidate because project 2a5f5458e52a43e78494327f7ed78f19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:22:48.392556 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-a1d793fb-c7b9-43c7-9afa-39007b411ab8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:48.396615 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-a1d793fb-c7b9-43c7-9afa-39007b411ab8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:48.400488 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1091/2198] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:22:47 2022] DELETE /identity/v3/projects/2a5f5458e52a43e78494327f7ed78f19 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 02 19:22:48.404820 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f828393-b861-4eb8-9582-bc5cac533ebc None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:48.407154 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-9f828393-b861-4eb8-9582-bc5cac533ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:48.424755 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f828393-b861-4eb8-9582-bc5cac533ebc None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:48.503363 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c4146-cf6a-4950-85ae-29bf712f375f None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:48.506779 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c4146-cf6a-4950-85ae-29bf712f375f None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:48.506779 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c4146-cf6a-4950-85ae-29bf712f375f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:48.533058 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460f3d5b-b6b2-4f1e-b531-07e85cc7f16a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:48.535644 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460f3d5b-b6b2-4f1e-b531-07e85cc7f16a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:48.535966 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460f3d5b-b6b2-4f1e-b531-07e85cc7f16a admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:48.536305 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460f3d5b-b6b2-4f1e-b531-07e85cc7f16a admin admin] PATH_INFO: `/v3/users/e2cb2cdf40fe4a6f94a7168d19754ff4` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:48.540765 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460f3d5b-b6b2-4f1e-b531-07e85cc7f16a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2cb2cdf40fe4a6f94a7168d19754ff4)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:48.544050 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460f3d5b-b6b2-4f1e-b531-07e85cc7f16a admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:48.563858 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-248c4146-cf6a-4950-85ae-29bf712f375f None None] MFA Rules not processed for user `e98d8cc46eb84774bc0ddaf8f1b17690`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:22:48.565110 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-248c4146-cf6a-4950-85ae-29bf712f375f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:48.640160 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:22:48.717484 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1092/2199] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:22:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 19:22:48.731404 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248c4146-cf6a-4950-85ae-29bf712f375f None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:48.733037 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-248c4146-cf6a-4950-85ae-29bf712f375f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:48.762251 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:22:48.851556 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248c4146-cf6a-4950-85ae-29bf712f375f None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:48.857006 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f9dc23-aaf0-437d-9631-071558f5a49a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:48.858033 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f9dc23-aaf0-437d-9631-071558f5a49a service cinder] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:48.858218 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f9dc23-aaf0-437d-9631-071558f5a49a service cinder] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:48.858481 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f9dc23-aaf0-437d-9631-071558f5a49a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:48.871536 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f9dc23-aaf0-437d-9631-071558f5a49a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:48.873949 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f9dc23-aaf0-437d-9631-071558f5a49a service cinder] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:48.876583 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-460f3d5b-b6b2-4f1e-b531-07e85cc7f16a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2cb2cdf40fe4a6f94a7168d19754ff4'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:48.889162 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1093/2200] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:22:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:22:48.894470 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-460f3d5b-b6b2-4f1e-b531-07e85cc7f16a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2cb2cdf40fe4a6f94a7168d19754ff4'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:49.022249 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-460f3d5b-b6b2-4f1e-b531-07e85cc7f16a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2cb2cdf40fe4a6f94a7168d19754ff4'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:49.026730 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-460f3d5b-b6b2-4f1e-b531-07e85cc7f16a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2cb2cdf40fe4a6f94a7168d19754ff4'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:49.026730 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-460f3d5b-b6b2-4f1e-b531-07e85cc7f16a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2cb2cdf40fe4a6f94a7168d19754ff4'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:49.062430 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1108/2201] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:22:48 2022] DELETE /identity/v3/users/e2cb2cdf40fe4a6f94a7168d19754ff4 => generated 0 bytes in 659 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:22:49.717812 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f9dc23-aaf0-437d-9631-071558f5a49a service cinder] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:49.725906 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-22f9dc23-aaf0-437d-9631-071558f5a49a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:49.772177 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f9dc23-aaf0-437d-9631-071558f5a49a service cinder] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:49.904264 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9a56a8-2dd4-4fda-9905-6c8b899da5f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:49.905935 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a56a8-2dd4-4fda-9905-6c8b899da5f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:49.906657 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a56a8-2dd4-4fda-9905-6c8b899da5f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:49.910064 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a56a8-2dd4-4fda-9905-6c8b899da5f2 admin admin] PATH_INFO: `/v3/projects/eea283c6fd8940f3be3c840217f0baa0` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:49.910330 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9a56a8-2dd4-4fda-9905-6c8b899da5f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eea283c6fd8940f3be3c840217f0baa0)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:49.914207 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9a56a8-2dd4-4fda-9905-6c8b899da5f2 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:50.050579 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-4a9a56a8-2dd4-4fda-9905-6c8b899da5f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eea283c6fd8940f3be3c840217f0baa0', 'request_id': 'req-4a9a56a8-2dd4-4fda-9905-6c8b899da5f2'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:50.159891 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-4a9a56a8-2dd4-4fda-9905-6c8b899da5f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eea283c6fd8940f3be3c840217f0baa0', 'request_id': 'req-4a9a56a8-2dd4-4fda-9905-6c8b899da5f2'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:50.168362 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-4a9a56a8-2dd4-4fda-9905-6c8b899da5f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eea283c6fd8940f3be3c840217f0baa0', 'request_id': 'req-4a9a56a8-2dd4-4fda-9905-6c8b899da5f2'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:50.186919 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-4a9a56a8-2dd4-4fda-9905-6c8b899da5f2 admin admin] The token cache is being invalidate because project eea283c6fd8940f3be3c840217f0baa0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:22:50.187402 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-4a9a56a8-2dd4-4fda-9905-6c8b899da5f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:50.188493 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-4a9a56a8-2dd4-4fda-9905-6c8b899da5f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:50.200008 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1094/2202] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:22:49 2022] DELETE /identity/v3/projects/eea283c6fd8940f3be3c840217f0baa0 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:22:50.202408 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89783489-6a8f-4b51-abbd-492111f4bbe3 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:50.203865 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89783489-6a8f-4b51-abbd-492111f4bbe3 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:50.204453 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89783489-6a8f-4b51-abbd-492111f4bbe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:50.232427 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-89783489-6a8f-4b51-abbd-492111f4bbe3 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:22:50.233853 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-89783489-6a8f-4b51-abbd-492111f4bbe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:50.336005 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1109/2203] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:22:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:22:50.343815 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1bc200-cf3e-4f09-a7c4-512543b55df7 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:50.348412 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1bc200-cf3e-4f09-a7c4-512543b55df7 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:50.348412 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1bc200-cf3e-4f09-a7c4-512543b55df7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:50.370588 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-0e1bc200-cf3e-4f09-a7c4-512543b55df7 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:22:50.372096 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-0e1bc200-cf3e-4f09-a7c4-512543b55df7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:50.388701 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1095/2204] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:22:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:22:50.395836 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89783489-6a8f-4b51-abbd-492111f4bbe3 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:50.413595 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89783489-6a8f-4b51-abbd-492111f4bbe3 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:50.421458 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9334bbae-816e-4a1d-bb19-7b25c730a158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:50.422879 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9334bbae-816e-4a1d-bb19-7b25c730a158 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:50.422879 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9334bbae-816e-4a1d-bb19-7b25c730a158 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:50.423094 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9334bbae-816e-4a1d-bb19-7b25c730a158 admin admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:50.424123 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9334bbae-816e-4a1d-bb19-7b25c730a158 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:22:50.424484 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9334bbae-816e-4a1d-bb19-7b25c730a158 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:50.427182 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9334bbae-816e-4a1d-bb19-7b25c730a158 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:50.446390 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1110/2205] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 2 19:22:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:22:50.451846 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1bc200-cf3e-4f09-a7c4-512543b55df7 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:50.469014 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1bc200-cf3e-4f09-a7c4-512543b55df7 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:50.476114 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314d46c9-2744-4b8c-923b-44518460ea0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:50.477253 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d46c9-2744-4b8c-923b-44518460ea0f admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:50.477747 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d46c9-2744-4b8c-923b-44518460ea0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:50.478253 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d46c9-2744-4b8c-923b-44518460ea0f admin admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:50.479639 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314d46c9-2744-4b8c-923b-44518460ea0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:50.483530 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314d46c9-2744-4b8c-923b-44518460ea0f admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:50.676687 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1096/2206] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:22:50 2022] POST /identity/v3/projects => generated 379 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:22:50.679055 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9334bbae-816e-4a1d-bb19-7b25c730a158 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:50.692445 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9334bbae-816e-4a1d-bb19-7b25c730a158 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:50.776203 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4699a9-38ce-49a8-9b53-ccdd90055f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:50.777338 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4699a9-38ce-49a8-9b53-ccdd90055f8a admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:50.777833 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4699a9-38ce-49a8-9b53-ccdd90055f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:50.778331 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4699a9-38ce-49a8-9b53-ccdd90055f8a admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:50.779650 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4699a9-38ce-49a8-9b53-ccdd90055f8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:50.785837 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4699a9-38ce-49a8-9b53-ccdd90055f8a admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:50.902801 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1111/2207] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:22:50 2022] POST /identity/v3/users => generated 290 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 02 19:22:50.905623 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314d46c9-2744-4b8c-923b-44518460ea0f admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:50.922809 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314d46c9-2744-4b8c-923b-44518460ea0f admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:50.928293 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7333b6c-6ca2-4a33-a2f4-45bdb6af77bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:50.928999 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7333b6c-6ca2-4a33-a2f4-45bdb6af77bb admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:50.929289 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7333b6c-6ca2-4a33-a2f4-45bdb6af77bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:50.929565 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7333b6c-6ca2-4a33-a2f4-45bdb6af77bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:50.930313 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7333b6c-6ca2-4a33-a2f4-45bdb6af77bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:50.931986 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7333b6c-6ca2-4a33-a2f4-45bdb6af77bb admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:50.938670 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:22:50.945177 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1097/2208] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:22:50 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:22:50.950170 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4699a9-38ce-49a8-9b53-ccdd90055f8a admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:50.961516 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4699a9-38ce-49a8-9b53-ccdd90055f8a admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:50.966510 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d98fd9-429f-4433-8152-26293e9951b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:50.967167 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d98fd9-429f-4433-8152-26293e9951b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:50.967459 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d98fd9-429f-4433-8152-26293e9951b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:50.967744 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d98fd9-429f-4433-8152-26293e9951b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:50.968497 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d98fd9-429f-4433-8152-26293e9951b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:50.970290 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d98fd9-429f-4433-8152-26293e9951b1 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:50.986611 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1112/2209] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:22:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:22:50.988371 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7333b6c-6ca2-4a33-a2f4-45bdb6af77bb admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:51.001686 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7333b6c-6ca2-4a33-a2f4-45bdb6af77bb admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:51.009959 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c2f7ed-e48b-40f8-95a3-9e9ab56aec96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:51.010924 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c2f7ed-e48b-40f8-95a3-9e9ab56aec96 admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:51.011215 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c2f7ed-e48b-40f8-95a3-9e9ab56aec96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:51.011501 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c2f7ed-e48b-40f8-95a3-9e9ab56aec96 admin admin] PATH_INFO: `/v3/projects/02fbb18b37a0438eab5c9b38cfc5e9d9/users/23b35eccd8714b08abcb54b65111f4d5/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:51.028329 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c2f7ed-e48b-40f8-95a3-9e9ab56aec96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02fbb18b37a0438eab5c9b38cfc5e9d9, user_id=23b35eccd8714b08abcb54b65111f4d5, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:51.031061 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c2f7ed-e48b-40f8-95a3-9e9ab56aec96 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:51.143006 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1098/2210] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:22:50 2022] PUT /identity/v3/projects/02fbb18b37a0438eab5c9b38cfc5e9d9/users/23b35eccd8714b08abcb54b65111f4d5/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:22:51.217027 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df88bd-2208-430a-8b69-faa41191f065 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:51.217635 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df88bd-2208-430a-8b69-faa41191f065 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:51.217635 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df88bd-2208-430a-8b69-faa41191f065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:51.226707 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:22:51.249329 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-42df88bd-2208-430a-8b69-faa41191f065 None None] MFA Rules not processed for user `23b35eccd8714b08abcb54b65111f4d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:22:51.250670 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-42df88bd-2208-430a-8b69-faa41191f065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:51.325998 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1113/2211] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:22:51 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:22:51.330948 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c2f7ed-e48b-40f8-95a3-9e9ab56aec96 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:51.402125 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c2f7ed-e48b-40f8-95a3-9e9ab56aec96 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:51.408154 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e308687-4bb2-473c-8a3f-f718b0483049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:51.409063 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e308687-4bb2-473c-8a3f-f718b0483049 admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:51.409468 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e308687-4bb2-473c-8a3f-f718b0483049 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:51.409843 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e308687-4bb2-473c-8a3f-f718b0483049 admin admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:51.411635 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e308687-4bb2-473c-8a3f-f718b0483049 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:51.414124 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e308687-4bb2-473c-8a3f-f718b0483049 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:51.586400 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1099/2212] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:22:51 2022] POST /identity/v3/projects => generated 381 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:22:51.607978 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42df88bd-2208-430a-8b69-faa41191f065 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:51.619643 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42df88bd-2208-430a-8b69-faa41191f065 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:51.687710 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd457fc2-f7b1-4d2e-b217-ea77cce0547d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:51.688568 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd457fc2-f7b1-4d2e-b217-ea77cce0547d admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:51.688939 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd457fc2-f7b1-4d2e-b217-ea77cce0547d admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:51.689340 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd457fc2-f7b1-4d2e-b217-ea77cce0547d admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:51.690193 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd457fc2-f7b1-4d2e-b217-ea77cce0547d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:51.692598 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd457fc2-f7b1-4d2e-b217-ea77cce0547d admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:51.775233 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1114/2213] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:22:51 2022] POST /identity/v3/users => generated 297 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 02 19:22:51.781107 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e308687-4bb2-473c-8a3f-f718b0483049 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:51.801060 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e308687-4bb2-473c-8a3f-f718b0483049 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:51.811873 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7113e0-9b0c-4c92-8395-ce08113808df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:51.813189 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7113e0-9b0c-4c92-8395-ce08113808df admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:51.813773 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7113e0-9b0c-4c92-8395-ce08113808df admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:51.814477 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7113e0-9b0c-4c92-8395-ce08113808df admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:51.816041 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7113e0-9b0c-4c92-8395-ce08113808df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:51.820002 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7113e0-9b0c-4c92-8395-ce08113808df admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:51.853163 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1100/2214] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:22:51 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 02 19:22:51.855157 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd457fc2-f7b1-4d2e-b217-ea77cce0547d admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:51.888608 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd457fc2-f7b1-4d2e-b217-ea77cce0547d admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:51.904280 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff13a9-e654-476c-afa5-a3bfc1c34f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:51.906364 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff13a9-e654-476c-afa5-a3bfc1c34f42 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:51.907251 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff13a9-e654-476c-afa5-a3bfc1c34f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:51.907969 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff13a9-e654-476c-afa5-a3bfc1c34f42 admin admin] PATH_INFO: `/v3/projects/fd7043c1edae4dc6a9cd17441a5bd3e5/users/a58c813a8fbd40c085b60035052466d8/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:51.947454 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ff13a9-e654-476c-afa5-a3bfc1c34f42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd7043c1edae4dc6a9cd17441a5bd3e5, user_id=a58c813a8fbd40c085b60035052466d8, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:51.954609 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ff13a9-e654-476c-afa5-a3bfc1c34f42 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:52.038911 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1115/2215] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:22:51 2022] PUT /identity/v3/projects/fd7043c1edae4dc6a9cd17441a5bd3e5/users/a58c813a8fbd40c085b60035052466d8/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:22:52.042524 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7113e0-9b0c-4c92-8395-ce08113808df admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:52.062042 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7113e0-9b0c-4c92-8395-ce08113808df admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:52.166546 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7cc2ac-618e-4dfd-845d-d2d6ae0a0b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:52.167455 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7cc2ac-618e-4dfd-845d-d2d6ae0a0b4b admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:52.168019 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7cc2ac-618e-4dfd-845d-d2d6ae0a0b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:52.168481 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7cc2ac-618e-4dfd-845d-d2d6ae0a0b4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:52.169886 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7cc2ac-618e-4dfd-845d-d2d6ae0a0b4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:52.173097 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7cc2ac-618e-4dfd-845d-d2d6ae0a0b4b admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:52.195576 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1101/2216] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:22:52 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:22:52.202003 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff13a9-e654-476c-afa5-a3bfc1c34f42 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:52.223662 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff13a9-e654-476c-afa5-a3bfc1c34f42 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:52.234964 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae0f5c8-f166-4c43-a0f2-10cd6703bae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:52.236889 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae0f5c8-f166-4c43-a0f2-10cd6703bae8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:52.237471 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae0f5c8-f166-4c43-a0f2-10cd6703bae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:52.238048 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae0f5c8-f166-4c43-a0f2-10cd6703bae8 admin admin] PATH_INFO: `/v3/projects/fd7043c1edae4dc6a9cd17441a5bd3e5/users/a58c813a8fbd40c085b60035052466d8/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:52.243261 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae0f5c8-f166-4c43-a0f2-10cd6703bae8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd7043c1edae4dc6a9cd17441a5bd3e5, user_id=a58c813a8fbd40c085b60035052466d8, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:52.250974 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae0f5c8-f166-4c43-a0f2-10cd6703bae8 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:52.279423 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1116/2217] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:22:52 2022] PUT /identity/v3/projects/fd7043c1edae4dc6a9cd17441a5bd3e5/users/a58c813a8fbd40c085b60035052466d8/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 02 19:22:52.298440 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c1f4f-cca6-43fd-bb66-f5c717a6f25a None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:52.298889 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c1f4f-cca6-43fd-bb66-f5c717a6f25a None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:52.299382 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c1f4f-cca6-43fd-bb66-f5c717a6f25a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:52.355601 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-c71c1f4f-cca6-43fd-bb66-f5c717a6f25a None None] MFA Rules not processed for user `a58c813a8fbd40c085b60035052466d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:22:52.357269 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-c71c1f4f-cca6-43fd-bb66-f5c717a6f25a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:52.428887 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:22:52.488015 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1102/2218] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:22:52 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:22:52.507835 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae0f5c8-f166-4c43-a0f2-10cd6703bae8 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:52.511205 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-0ae0f5c8-f166-4c43-a0f2-10cd6703bae8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:52.603332 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:22:52.613508 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae0f5c8-f166-4c43-a0f2-10cd6703bae8 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:52.620632 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63a90bc-fa2e-4087-ad7f-fe8a7601c1da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:52.621728 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a90bc-fa2e-4087-ad7f-fe8a7601c1da service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:52.622396 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a90bc-fa2e-4087-ad7f-fe8a7601c1da service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:52.622915 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a90bc-fa2e-4087-ad7f-fe8a7601c1da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:52.640120 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63a90bc-fa2e-4087-ad7f-fe8a7601c1da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:52.642525 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63a90bc-fa2e-4087-ad7f-fe8a7601c1da service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:52.666230 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1117/2219] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:22:52 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:22:53.161885 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f35b3-ad36-4eba-8a54-c1b3c0ef8d4e None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:53.163049 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f35b3-ad36-4eba-8a54-c1b3c0ef8d4e None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:53.163532 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f35b3-ad36-4eba-8a54-c1b3c0ef8d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:53.192324 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-a93f35b3-ad36-4eba-8a54-c1b3c0ef8d4e None None] MFA Rules not processed for user `e98d8cc46eb84774bc0ddaf8f1b17690`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:22:53.194185 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-a93f35b3-ad36-4eba-8a54-c1b3c0ef8d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:53.285148 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1103/2220] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:22:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:22:53.325238 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9236a1-59af-4418-9b81-6aabce042742 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:53.325610 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9236a1-59af-4418-9b81-6aabce042742 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:53.326068 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9236a1-59af-4418-9b81-6aabce042742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:53.371859 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-0a9236a1-59af-4418-9b81-6aabce042742 None None] MFA Rules not processed for user `a0681f3eff324262b4289bf6d8f4c311`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:22:53.375915 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-0a9236a1-59af-4418-9b81-6aabce042742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:53.497971 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1118/2221] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:22:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 19:22:53.652530 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93f35b3-ad36-4eba-8a54-c1b3c0ef8d4e None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:53.811120 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93f35b3-ad36-4eba-8a54-c1b3c0ef8d4e None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:53.813917 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbe6747-2c2a-462f-bbc1-903484c4b276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:53.814854 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe6747-2c2a-462f-bbc1-903484c4b276 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:53.815147 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe6747-2c2a-462f-bbc1-903484c4b276 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:53.815428 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe6747-2c2a-462f-bbc1-903484c4b276 admin admin] PATH_INFO: `/v3/users/23b35eccd8714b08abcb54b65111f4d5` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:53.816747 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbe6747-2c2a-462f-bbc1-903484c4b276 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23b35eccd8714b08abcb54b65111f4d5)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:53.819346 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbe6747-2c2a-462f-bbc1-903484c4b276 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:54.081467 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-fcbe6747-2c2a-462f-bbc1-903484c4b276 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23b35eccd8714b08abcb54b65111f4d5'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:54.091348 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-fcbe6747-2c2a-462f-bbc1-903484c4b276 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23b35eccd8714b08abcb54b65111f4d5'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:54.092771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-fcbe6747-2c2a-462f-bbc1-903484c4b276 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23b35eccd8714b08abcb54b65111f4d5'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:54.194562 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-fcbe6747-2c2a-462f-bbc1-903484c4b276 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23b35eccd8714b08abcb54b65111f4d5'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:54.195991 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-fcbe6747-2c2a-462f-bbc1-903484c4b276 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23b35eccd8714b08abcb54b65111f4d5'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:54.211748 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:22:54.221429 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1104/2222] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:22:53 2022] DELETE /identity/v3/users/23b35eccd8714b08abcb54b65111f4d5 => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:22:54.226950 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c2fb05-bd31-459a-a269-1c568cd591c5 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:54.227332 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c2fb05-bd31-459a-a269-1c568cd591c5 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:54.227530 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c2fb05-bd31-459a-a269-1c568cd591c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:54.247603 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-58c2fb05-bd31-459a-a269-1c568cd591c5 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:22:54.248879 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-58c2fb05-bd31-459a-a269-1c568cd591c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:54.349489 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1119/2223] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:22:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 19:22:54.360802 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbe6747-2c2a-462f-bbc1-903484c4b276 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:54.368001 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-fcbe6747-2c2a-462f-bbc1-903484c4b276 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:54.506175 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbe6747-2c2a-462f-bbc1-903484c4b276 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:54.510488 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52577e2d-216c-486b-8407-5167d45f9516 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:54.511436 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52577e2d-216c-486b-8407-5167d45f9516 service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:54.511725 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52577e2d-216c-486b-8407-5167d45f9516 service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:54.512006 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52577e2d-216c-486b-8407-5167d45f9516 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:54.524702 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52577e2d-216c-486b-8407-5167d45f9516 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:54.526234 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52577e2d-216c-486b-8407-5167d45f9516 service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:54.538636 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1105/2224] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:22:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:22:55.442713 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c2fb05-bd31-459a-a269-1c568cd591c5 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:55.445457 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-58c2fb05-bd31-459a-a269-1c568cd591c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:55.475894 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c2fb05-bd31-459a-a269-1c568cd591c5 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:55.480807 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c7401-75fc-4fcf-88ae-5a51abdecb36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:55.482004 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c7401-75fc-4fcf-88ae-5a51abdecb36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:55.482291 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c7401-75fc-4fcf-88ae-5a51abdecb36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:55.482576 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c7401-75fc-4fcf-88ae-5a51abdecb36 admin admin] PATH_INFO: `/v3/projects/02fbb18b37a0438eab5c9b38cfc5e9d9` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:55.484046 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561c7401-75fc-4fcf-88ae-5a51abdecb36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02fbb18b37a0438eab5c9b38cfc5e9d9)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:55.486414 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561c7401-75fc-4fcf-88ae-5a51abdecb36 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:55.688853 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-561c7401-75fc-4fcf-88ae-5a51abdecb36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02fbb18b37a0438eab5c9b38cfc5e9d9', 'request_id': 'req-561c7401-75fc-4fcf-88ae-5a51abdecb36'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:55.777031 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-561c7401-75fc-4fcf-88ae-5a51abdecb36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02fbb18b37a0438eab5c9b38cfc5e9d9', 'request_id': 'req-561c7401-75fc-4fcf-88ae-5a51abdecb36'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:55.785482 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-561c7401-75fc-4fcf-88ae-5a51abdecb36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02fbb18b37a0438eab5c9b38cfc5e9d9', 'request_id': 'req-561c7401-75fc-4fcf-88ae-5a51abdecb36'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:55.812500 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-561c7401-75fc-4fcf-88ae-5a51abdecb36 admin admin] The token cache is being invalidate because project 02fbb18b37a0438eab5c9b38cfc5e9d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:22:55.812777 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-561c7401-75fc-4fcf-88ae-5a51abdecb36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:55.814338 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-561c7401-75fc-4fcf-88ae-5a51abdecb36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:55.817605 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1120/2225] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:22:55 2022] DELETE /identity/v3/projects/02fbb18b37a0438eab5c9b38cfc5e9d9 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:22:55.823357 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52577e2d-216c-486b-8407-5167d45f9516 service neutron] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:55.837365 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52577e2d-216c-486b-8407-5167d45f9516 service neutron] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:56.050094 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7510cd4a-8393-40da-83ac-4e6c8c460f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:56.051672 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7510cd4a-8393-40da-83ac-4e6c8c460f5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:56.053380 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7510cd4a-8393-40da-83ac-4e6c8c460f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:56.053380 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7510cd4a-8393-40da-83ac-4e6c8c460f5b admin admin] PATH_INFO: `/v3/users/a58c813a8fbd40c085b60035052466d8` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:56.062848 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7510cd4a-8393-40da-83ac-4e6c8c460f5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a58c813a8fbd40c085b60035052466d8)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:56.067163 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7510cd4a-8393-40da-83ac-4e6c8c460f5b admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:56.399656 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-7510cd4a-8393-40da-83ac-4e6c8c460f5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a58c813a8fbd40c085b60035052466d8'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:56.425537 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-7510cd4a-8393-40da-83ac-4e6c8c460f5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a58c813a8fbd40c085b60035052466d8'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:56.426723 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-7510cd4a-8393-40da-83ac-4e6c8c460f5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a58c813a8fbd40c085b60035052466d8'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:56.657651 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-7510cd4a-8393-40da-83ac-4e6c8c460f5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a58c813a8fbd40c085b60035052466d8'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:56.658695 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-7510cd4a-8393-40da-83ac-4e6c8c460f5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a58c813a8fbd40c085b60035052466d8'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:56.682866 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1106/2226] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:22:55 2022] DELETE /identity/v3/users/a58c813a8fbd40c085b60035052466d8 => generated 0 bytes in 860 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:22:57.353070 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c7401-75fc-4fcf-88ae-5a51abdecb36 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:57.354956 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-561c7401-75fc-4fcf-88ae-5a51abdecb36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:57.401316 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c7401-75fc-4fcf-88ae-5a51abdecb36 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:57.419483 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:22:57.493455 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c96507-42e0-4802-89d1-ce763bfe660c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:57.493455 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c96507-42e0-4802-89d1-ce763bfe660c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:57.493455 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c96507-42e0-4802-89d1-ce763bfe660c admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:57.493455 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c96507-42e0-4802-89d1-ce763bfe660c admin admin] PATH_INFO: `/v3/projects/fd7043c1edae4dc6a9cd17441a5bd3e5` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:57.494651 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c96507-42e0-4802-89d1-ce763bfe660c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd7043c1edae4dc6a9cd17441a5bd3e5)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:57.495556 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c96507-42e0-4802-89d1-ce763bfe660c admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:57.751443 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c4c96507-42e0-4802-89d1-ce763bfe660c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd7043c1edae4dc6a9cd17441a5bd3e5', 'request_id': 'req-c4c96507-42e0-4802-89d1-ce763bfe660c'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:57.874699 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c4c96507-42e0-4802-89d1-ce763bfe660c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd7043c1edae4dc6a9cd17441a5bd3e5', 'request_id': 'req-c4c96507-42e0-4802-89d1-ce763bfe660c'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:57.885769 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c4c96507-42e0-4802-89d1-ce763bfe660c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd7043c1edae4dc6a9cd17441a5bd3e5', 'request_id': 'req-c4c96507-42e0-4802-89d1-ce763bfe660c'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:57.911283 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c4c96507-42e0-4802-89d1-ce763bfe660c admin admin] The token cache is being invalidate because project fd7043c1edae4dc6a9cd17441a5bd3e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:22:57.911765 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c4c96507-42e0-4802-89d1-ce763bfe660c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:57.914038 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c4c96507-42e0-4802-89d1-ce763bfe660c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:22:57.916898 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1121/2227] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:22:57 2022] DELETE /identity/v3/projects/fd7043c1edae4dc6a9cd17441a5bd3e5 => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:22:57.925820 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581786e-fab2-4dac-8afd-cbed28658d65 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:57.927274 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581786e-fab2-4dac-8afd-cbed28658d65 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:57.928100 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581786e-fab2-4dac-8afd-cbed28658d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:57.971054 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-8581786e-fab2-4dac-8afd-cbed28658d65 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:22:57.972314 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-8581786e-fab2-4dac-8afd-cbed28658d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:57.977826 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:22:58.095706 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1107/2228] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:22:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:22:58.114538 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fddf8b-b83e-49f1-b0e1-4eade26525cb None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:58.115932 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fddf8b-b83e-49f1-b0e1-4eade26525cb None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:58.116577 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fddf8b-b83e-49f1-b0e1-4eade26525cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:58.163717 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-69fddf8b-b83e-49f1-b0e1-4eade26525cb None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:22:58.166414 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-69fddf8b-b83e-49f1-b0e1-4eade26525cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:58.199512 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1122/2229] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:22:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:22:58.205407 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8581786e-fab2-4dac-8afd-cbed28658d65 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:58.216583 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8581786e-fab2-4dac-8afd-cbed28658d65 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:58.222851 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7338a4dd-dd29-4012-94af-1e49add67c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:58.223561 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7338a4dd-dd29-4012-94af-1e49add67c85 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:58.223849 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7338a4dd-dd29-4012-94af-1e49add67c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:58.224134 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7338a4dd-dd29-4012-94af-1e49add67c85 admin admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:58.224950 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7338a4dd-dd29-4012-94af-1e49add67c85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:22:58.225457 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7338a4dd-dd29-4012-94af-1e49add67c85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:58.227398 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7338a4dd-dd29-4012-94af-1e49add67c85 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:58.243733 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1108/2230] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 2 19:22:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:22:58.250218 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fddf8b-b83e-49f1-b0e1-4eade26525cb None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:58.262375 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fddf8b-b83e-49f1-b0e1-4eade26525cb None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:58.267796 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b02dda1-67ac-4864-af2a-985dd526e00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:58.268496 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b02dda1-67ac-4864-af2a-985dd526e00d admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:58.268783 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b02dda1-67ac-4864-af2a-985dd526e00d admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:58.269067 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b02dda1-67ac-4864-af2a-985dd526e00d admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:58.269812 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b02dda1-67ac-4864-af2a-985dd526e00d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:58.272006 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b02dda1-67ac-4864-af2a-985dd526e00d admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:58.430487 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1123/2231] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:22:58 2022] POST /identity/v3/projects => generated 383 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 02 19:22:58.431301 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7338a4dd-dd29-4012-94af-1e49add67c85 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:58.446526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7338a4dd-dd29-4012-94af-1e49add67c85 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:58.576414 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9961a46-4ed3-4d71-b306-3d81717db67c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:58.577052 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9961a46-4ed3-4d71-b306-3d81717db67c admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:58.577340 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9961a46-4ed3-4d71-b306-3d81717db67c admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:58.577617 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9961a46-4ed3-4d71-b306-3d81717db67c admin admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:58.578343 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9961a46-4ed3-4d71-b306-3d81717db67c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:58.580375 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9961a46-4ed3-4d71-b306-3d81717db67c admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:58.694442 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1109/2232] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:22:58 2022] POST /identity/v3/users => generated 292 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 02 19:22:58.700562 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b02dda1-67ac-4864-af2a-985dd526e00d admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:58.718913 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b02dda1-67ac-4864-af2a-985dd526e00d admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:58.729422 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc037fa-5096-4c3a-9fde-bd574e696bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:58.731139 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc037fa-5096-4c3a-9fde-bd574e696bd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:58.732065 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc037fa-5096-4c3a-9fde-bd574e696bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:58.732769 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc037fa-5096-4c3a-9fde-bd574e696bd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:58.734406 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc037fa-5096-4c3a-9fde-bd574e696bd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:58.738194 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc037fa-5096-4c3a-9fde-bd574e696bd3 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:58.770796 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1124/2233] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:22:58 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:22:58.775415 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9961a46-4ed3-4d71-b306-3d81717db67c admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:58.786693 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9961a46-4ed3-4d71-b306-3d81717db67c admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:58.792594 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ade480-4f53-4a4d-93e1-e10fe3d9ff1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:58.793323 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ade480-4f53-4a4d-93e1-e10fe3d9ff1e admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:58.793609 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ade480-4f53-4a4d-93e1-e10fe3d9ff1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:58.793889 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ade480-4f53-4a4d-93e1-e10fe3d9ff1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:58.794650 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ade480-4f53-4a4d-93e1-e10fe3d9ff1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:58.796387 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ade480-4f53-4a4d-93e1-e10fe3d9ff1e admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:58.811137 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1110/2234] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:22:58 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:22:58.816411 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc037fa-5096-4c3a-9fde-bd574e696bd3 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:58.835366 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc037fa-5096-4c3a-9fde-bd574e696bd3 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:58.851088 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7fd01d-fdd3-49c5-a110-dc971bcea8a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:58.852307 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7fd01d-fdd3-49c5-a110-dc971bcea8a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:58.852641 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7fd01d-fdd3-49c5-a110-dc971bcea8a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:58.853037 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7fd01d-fdd3-49c5-a110-dc971bcea8a2 admin admin] PATH_INFO: `/v3/projects/55e8f00d4bd94b10b5dc1ce3615c337a/users/4a34594a145b40ed9b268ba2d66da3aa/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:58.880506 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7fd01d-fdd3-49c5-a110-dc971bcea8a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55e8f00d4bd94b10b5dc1ce3615c337a, user_id=4a34594a145b40ed9b268ba2d66da3aa, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:58.885819 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7fd01d-fdd3-49c5-a110-dc971bcea8a2 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:59.044593 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1125/2235] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:22:58 2022] PUT /identity/v3/projects/55e8f00d4bd94b10b5dc1ce3615c337a/users/4a34594a145b40ed9b268ba2d66da3aa/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:22:59.051252 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f117d-4e32-45ac-ae5c-493485cd6067 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:59.051868 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f117d-4e32-45ac-ae5c-493485cd6067 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:59.052324 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f117d-4e32-45ac-ae5c-493485cd6067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:59.087412 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-5e5f117d-4e32-45ac-ae5c-493485cd6067 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:22:59.090020 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-5e5f117d-4e32-45ac-ae5c-493485cd6067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:59.217715 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1111/2236] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:22:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 19:22:59.229979 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7fd01d-fdd3-49c5-a110-dc971bcea8a2 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:22:59.232227 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-4d7fd01d-fdd3-49c5-a110-dc971bcea8a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:22:59.293662 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:22:59.309744 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7fd01d-fdd3-49c5-a110-dc971bcea8a2 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:22:59.314228 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e9d107-843d-4e8c-91a6-3ebd133ac4f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:22:59.314944 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e9d107-843d-4e8c-91a6-3ebd133ac4f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:22:59.315236 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e9d107-843d-4e8c-91a6-3ebd133ac4f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:22:59.315521 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e9d107-843d-4e8c-91a6-3ebd133ac4f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:22:59.327203 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e9d107-843d-4e8c-91a6-3ebd133ac4f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:22:59.328722 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e9d107-843d-4e8c-91a6-3ebd133ac4f4 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:22:59.341146 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1126/2237] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:22:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 02 19:23:08.965560 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5f117d-4e32-45ac-ae5c-493485cd6067 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:23:08.973855 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-5e5f117d-4e32-45ac-ae5c-493485cd6067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:23:09.073748 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5f117d-4e32-45ac-ae5c-493485cd6067 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:23:09.095093 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a66d26-736e-43a1-bdee-510a2559b1e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:23:09.097003 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a66d26-736e-43a1-bdee-510a2559b1e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:23:09.097616 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a66d26-736e-43a1-bdee-510a2559b1e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:23:09.098713 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a66d26-736e-43a1-bdee-510a2559b1e0 admin admin] PATH_INFO: `/v3/users/92dd1cba1c1648f39730f9560e781b10` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:23:09.107952 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a66d26-736e-43a1-bdee-510a2559b1e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92dd1cba1c1648f39730f9560e781b10)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:23:09.130811 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a66d26-736e-43a1-bdee-510a2559b1e0 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:23:09.243806 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:23:09.566591 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edccfb4b-4e41-44e6-8cf4-efca7cbf0d94 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:23:09.567240 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edccfb4b-4e41-44e6-8cf4-efca7cbf0d94 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:23:09.567591 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edccfb4b-4e41-44e6-8cf4-efca7cbf0d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:23:09.659075 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-15a66d26-736e-43a1-bdee-510a2559b1e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92dd1cba1c1648f39730f9560e781b10'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:23:09.694920 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-15a66d26-736e-43a1-bdee-510a2559b1e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92dd1cba1c1648f39730f9560e781b10'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:23:09.702115 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-15a66d26-736e-43a1-bdee-510a2559b1e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92dd1cba1c1648f39730f9560e781b10'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:23:09.713619 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-edccfb4b-4e41-44e6-8cf4-efca7cbf0d94 None None] MFA Rules not processed for user `4a34594a145b40ed9b268ba2d66da3aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:23:09.715243 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-edccfb4b-4e41-44e6-8cf4-efca7cbf0d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:23:09.829679 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-15a66d26-736e-43a1-bdee-510a2559b1e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92dd1cba1c1648f39730f9560e781b10'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:23:09.831142 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-15a66d26-736e-43a1-bdee-510a2559b1e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92dd1cba1c1648f39730f9560e781b10'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:23:09.884865 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1112/2238] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:23:08 2022] DELETE /identity/v3/users/92dd1cba1c1648f39730f9560e781b10 => generated 0 bytes in 923 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:23:09.892080 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b217d37-34a2-4839-b010-0a5f66f50513 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:23:09.893116 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b217d37-34a2-4839-b010-0a5f66f50513 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:23:09.895791 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b217d37-34a2-4839-b010-0a5f66f50513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:23:09.909016 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1127/2239] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:23:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:23:09.910632 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edccfb4b-4e41-44e6-8cf4-efca7cbf0d94 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:23:09.919879 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-edccfb4b-4e41-44e6-8cf4-efca7cbf0d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:23:09.939077 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-1b217d37-34a2-4839-b010-0a5f66f50513 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:23:09.944124 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-1b217d37-34a2-4839-b010-0a5f66f50513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:23:09.952152 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edccfb4b-4e41-44e6-8cf4-efca7cbf0d94 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:23:10.049793 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb551d99-43df-463a-8e5b-91d22d89509e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:23:10.051724 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb551d99-43df-463a-8e5b-91d22d89509e admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:23:10.052232 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb551d99-43df-463a-8e5b-91d22d89509e admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:23:10.052654 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb551d99-43df-463a-8e5b-91d22d89509e admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:23:10.053580 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb551d99-43df-463a-8e5b-91d22d89509e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:23:10.056404 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb551d99-43df-463a-8e5b-91d22d89509e admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:23:10.092328 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1113/2240] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:23:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:23:10.104395 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b217d37-34a2-4839-b010-0a5f66f50513 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:23:10.107202 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-1b217d37-34a2-4839-b010-0a5f66f50513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:23:10.163809 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1128/2241] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:23:09 2022] POST /identity/v3/projects => generated 383 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:23:10.168220 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb551d99-43df-463a-8e5b-91d22d89509e admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:23:10.303981 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b217d37-34a2-4839-b010-0a5f66f50513 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:23:10.363652 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb551d99-43df-463a-8e5b-91d22d89509e admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:23:10.372171 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39d48c1-9685-4eb8-8927-22123daceec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:23:10.373345 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39d48c1-9685-4eb8-8927-22123daceec8 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:23:10.373841 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39d48c1-9685-4eb8-8927-22123daceec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:23:10.374378 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39d48c1-9685-4eb8-8927-22123daceec8 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:23:10.376486 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39d48c1-9685-4eb8-8927-22123daceec8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:23:10.380283 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39d48c1-9685-4eb8-8927-22123daceec8 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:23:10.393474 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:23:10.400735 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46737972-8803-4f52-bc3b-76bd09d11411 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:23:10.401656 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46737972-8803-4f52-bc3b-76bd09d11411 service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:23:10.401851 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46737972-8803-4f52-bc3b-76bd09d11411 service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:23:10.402223 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46737972-8803-4f52-bc3b-76bd09d11411 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:23:10.413734 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46737972-8803-4f52-bc3b-76bd09d11411 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:23:10.415365 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46737972-8803-4f52-bc3b-76bd09d11411 service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:23:10.437886 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1114/2242] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:23:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:23:10.521332 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1129/2243] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:23:10 2022] POST /identity/v3/users => generated 298 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:23:10.523060 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:23:10.525969 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46737972-8803-4f52-bc3b-76bd09d11411 service neutron] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:23:10.536983 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46737972-8803-4f52-bc3b-76bd09d11411 service neutron] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:23:10.542468 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8199996f-faec-43b2-a7ac-1f84d5b918d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:23:10.543338 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199996f-faec-43b2-a7ac-1f84d5b918d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:23:10.543650 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199996f-faec-43b2-a7ac-1f84d5b918d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:23:10.543958 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199996f-faec-43b2-a7ac-1f84d5b918d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:23:10.544745 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8199996f-faec-43b2-a7ac-1f84d5b918d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:23:10.546686 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8199996f-faec-43b2-a7ac-1f84d5b918d9 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:23:10.558976 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1115/2244] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:23:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:23:10.563805 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39d48c1-9685-4eb8-8927-22123daceec8 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:23:10.579961 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39d48c1-9685-4eb8-8927-22123daceec8 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:23:10.589750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c05301c-ae9b-4e38-83bc-10859f76fc19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:23:10.591388 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c05301c-ae9b-4e38-83bc-10859f76fc19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:23:10.592133 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c05301c-ae9b-4e38-83bc-10859f76fc19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:23:10.592912 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c05301c-ae9b-4e38-83bc-10859f76fc19 admin admin] PATH_INFO: `/v3/projects/23a8a3c4656049d69941aca690dd35d3/users/00fe81e5d6584911a78ff1d440f2f8c7/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:23:10.616364 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c05301c-ae9b-4e38-83bc-10859f76fc19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23a8a3c4656049d69941aca690dd35d3, user_id=00fe81e5d6584911a78ff1d440f2f8c7, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:23:10.620018 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c05301c-ae9b-4e38-83bc-10859f76fc19 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:23:10.728627 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1130/2245] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:23:10 2022] PUT /identity/v3/projects/23a8a3c4656049d69941aca690dd35d3/users/00fe81e5d6584911a78ff1d440f2f8c7/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 02 19:23:10.730060 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8199996f-faec-43b2-a7ac-1f84d5b918d9 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:23:10.766196 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8199996f-faec-43b2-a7ac-1f84d5b918d9 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:23:10.854674 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d429b99f-9b1c-4f72-a9a9-5ef36a03793a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:23:10.855922 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d429b99f-9b1c-4f72-a9a9-5ef36a03793a admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:23:10.856412 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d429b99f-9b1c-4f72-a9a9-5ef36a03793a admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:23:10.856921 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d429b99f-9b1c-4f72-a9a9-5ef36a03793a admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:23:10.858394 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d429b99f-9b1c-4f72-a9a9-5ef36a03793a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:23:10.861780 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d429b99f-9b1c-4f72-a9a9-5ef36a03793a admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:23:10.884030 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1116/2246] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:23:10 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:23:10.888672 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c05301c-ae9b-4e38-83bc-10859f76fc19 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:23:10.902384 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c05301c-ae9b-4e38-83bc-10859f76fc19 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:23:10.912536 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcde891-5a13-4330-8407-7d257e1c26cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:23:10.913661 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcde891-5a13-4330-8407-7d257e1c26cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:23:10.914069 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcde891-5a13-4330-8407-7d257e1c26cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:23:10.914359 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcde891-5a13-4330-8407-7d257e1c26cf admin admin] PATH_INFO: `/v3/projects/23a8a3c4656049d69941aca690dd35d3/users/00fe81e5d6584911a78ff1d440f2f8c7/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:23:10.916864 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcde891-5a13-4330-8407-7d257e1c26cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=23a8a3c4656049d69941aca690dd35d3, user_id=00fe81e5d6584911a78ff1d440f2f8c7, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:23:10.919828 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcde891-5a13-4330-8407-7d257e1c26cf admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:23:10.937293 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1131/2247] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:23:10 2022] PUT /identity/v3/projects/23a8a3c4656049d69941aca690dd35d3/users/00fe81e5d6584911a78ff1d440f2f8c7/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:23:11.086830 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d429b99f-9b1c-4f72-a9a9-5ef36a03793a admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:23:11.090348 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-d429b99f-9b1c-4f72-a9a9-5ef36a03793a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:23:11.133456 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d429b99f-9b1c-4f72-a9a9-5ef36a03793a admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:23:11.210841 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a6ed24-a992-42ba-966c-1479bf7b08e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:23:11.212076 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a6ed24-a992-42ba-966c-1479bf7b08e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:23:11.212076 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a6ed24-a992-42ba-966c-1479bf7b08e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:23:11.212076 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a6ed24-a992-42ba-966c-1479bf7b08e8 admin admin] PATH_INFO: `/v3/projects/0e3db6d00eed46f38a66fe513e0ac1a3` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:23:11.213455 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a6ed24-a992-42ba-966c-1479bf7b08e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e3db6d00eed46f38a66fe513e0ac1a3)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:23:11.215688 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a6ed24-a992-42ba-966c-1479bf7b08e8 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:23:11.391445 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-11a6ed24-a992-42ba-966c-1479bf7b08e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e3db6d00eed46f38a66fe513e0ac1a3', 'request_id': 'req-11a6ed24-a992-42ba-966c-1479bf7b08e8'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:23:11.520025 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-11a6ed24-a992-42ba-966c-1479bf7b08e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e3db6d00eed46f38a66fe513e0ac1a3', 'request_id': 'req-11a6ed24-a992-42ba-966c-1479bf7b08e8'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:23:11.531633 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-11a6ed24-a992-42ba-966c-1479bf7b08e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e3db6d00eed46f38a66fe513e0ac1a3', 'request_id': 'req-11a6ed24-a992-42ba-966c-1479bf7b08e8'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:23:11.550634 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-11a6ed24-a992-42ba-966c-1479bf7b08e8 admin admin] The token cache is being invalidate because project 0e3db6d00eed46f38a66fe513e0ac1a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:23:11.551148 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-11a6ed24-a992-42ba-966c-1479bf7b08e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:23:11.551982 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-11a6ed24-a992-42ba-966c-1479bf7b08e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:23:11.556424 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1117/2248] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:23:11 2022] DELETE /identity/v3/projects/0e3db6d00eed46f38a66fe513e0ac1a3 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:23:21.378000 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:INSERT=4,keystone:SELECT=17 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:23:21.972186 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:23:22.368620 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f351188-cd96-4d5b-a27a-6638fb0b8cba None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:23:22.368784 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f351188-cd96-4d5b-a27a-6638fb0b8cba None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:23:22.369267 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f351188-cd96-4d5b-a27a-6638fb0b8cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:23:22.432640 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-0f351188-cd96-4d5b-a27a-6638fb0b8cba None None] MFA Rules not processed for user `00fe81e5d6584911a78ff1d440f2f8c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:23:22.435198 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-0f351188-cd96-4d5b-a27a-6638fb0b8cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:23:22.673701 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1132/2249] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:23:22 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:23:22.694916 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a6ed24-a992-42ba-966c-1479bf7b08e8 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:23:22.698311 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-11a6ed24-a992-42ba-966c-1479bf7b08e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:23:22.840029 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a6ed24-a992-42ba-966c-1479bf7b08e8 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:23:22.846658 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cf5709-e4fd-4db4-ac6b-cd82ce4fd624 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07892c0aa21449b94a1cc78e3b8a653', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:23:22.849493 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf5709-e4fd-4db4-ac6b-cd82ce4fd624 service glance] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:23:22.853818 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf5709-e4fd-4db4-ac6b-cd82ce4fd624 service glance] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:23:22.853818 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf5709-e4fd-4db4-ac6b-cd82ce4fd624 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:23:22.854675 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-22cf5709-e4fd-4db4-ac6b-cd82ce4fd624 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:23:22.980145 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cf5709-e4fd-4db4-ac6b-cd82ce4fd624 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:23:22.982850 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cf5709-e4fd-4db4-ac6b-cd82ce4fd624 service glance] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:23:22.998406 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1118/2250] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:23:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:23:23.364834 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dba7ab-9d05-4a5b-9591-dd324013bd76 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:23:23.365244 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dba7ab-9d05-4a5b-9591-dd324013bd76 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:23:23.365790 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dba7ab-9d05-4a5b-9591-dd324013bd76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:23:23.401814 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-c5dba7ab-9d05-4a5b-9591-dd324013bd76 None None] MFA Rules not processed for user `04f64813f09348eb91cb7bb2869f7612`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:23:23.403412 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-c5dba7ab-9d05-4a5b-9591-dd324013bd76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:23:23.480866 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1133/2251] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Jun 2 19:23:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 19:23:23.497161 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cf5709-e4fd-4db4-ac6b-cd82ce4fd624 service glance] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:23:23.498892 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-22cf5709-e4fd-4db4-ac6b-cd82ce4fd624 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:23:23.606163 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cf5709-e4fd-4db4-ac6b-cd82ce4fd624 service glance] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:23:23.616067 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f964bb-3abe-440f-bc02-a00d57640e2c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a59e1d67ff94186a3ebcfcb49938a87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:23:23.617167 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f964bb-3abe-440f-bc02-a00d57640e2c service swift] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:23:23.617681 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f964bb-3abe-440f-bc02-a00d57640e2c service swift] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:23:23.618490 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f964bb-3abe-440f-bc02-a00d57640e2c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:23:23.633805 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f964bb-3abe-440f-bc02-a00d57640e2c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:23:23.636462 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f964bb-3abe-440f-bc02-a00d57640e2c service swift] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:23:23.650857 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1119/2252] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 2 19:23:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 02 19:23:33.472866 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=66 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:23:33.646035 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=80 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:23:38.110840 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf1cf3-1842-4e55-a427-1eb6ddbd8e0a None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:23:38.112129 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf1cf3-1842-4e55-a427-1eb6ddbd8e0a None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:23:38.112716 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf1cf3-1842-4e55-a427-1eb6ddbd8e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:23:38.144753 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-e3bf1cf3-1842-4e55-a427-1eb6ddbd8e0a None None] MFA Rules not processed for user `4a34594a145b40ed9b268ba2d66da3aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:23:38.147413 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-e3bf1cf3-1842-4e55-a427-1eb6ddbd8e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:23:38.159522 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1134/2253] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:23:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:23:42.455936 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14114191-f9a4-4af1-abdb-78c842a6d9b0 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:23:42.456802 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14114191-f9a4-4af1-abdb-78c842a6d9b0 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:23:42.456802 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14114191-f9a4-4af1-abdb-78c842a6d9b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:23:42.546723 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-14114191-f9a4-4af1-abdb-78c842a6d9b0 None None] MFA Rules not processed for user `04f64813f09348eb91cb7bb2869f7612`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:23:42.549165 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-14114191-f9a4-4af1-abdb-78c842a6d9b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:23:42.581100 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1120/2254] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Jun 2 19:23:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 19:23:42.601568 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bf1cf3-1842-4e55-a427-1eb6ddbd8e0a None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:23:42.654114 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bf1cf3-1842-4e55-a427-1eb6ddbd8e0a None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:23:42.663477 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38868688-d420-433e-b091-03cebfcae5ac service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a59e1d67ff94186a3ebcfcb49938a87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:23:42.664276 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38868688-d420-433e-b091-03cebfcae5ac service swift] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:23:42.664667 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38868688-d420-433e-b091-03cebfcae5ac service swift] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:23:42.665153 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38868688-d420-433e-b091-03cebfcae5ac service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:23:42.679014 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38868688-d420-433e-b091-03cebfcae5ac service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:23:42.682379 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38868688-d420-433e-b091-03cebfcae5ac service swift] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:23:42.705029 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1135/2255] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 2 19:23:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 02 19:23:52.576671 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=18 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:23:52.699684 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=13 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:23:57.788400 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14114191-f9a4-4af1-abdb-78c842a6d9b0 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:23:57.790740 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-14114191-f9a4-4af1-abdb-78c842a6d9b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:23:57.907356 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14114191-f9a4-4af1-abdb-78c842a6d9b0 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:23:57.916914 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137276c8-7e16-4d2e-8b8a-56ad1b40d110 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73ae837e99764bebbe7a6e06ac9049e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:23:57.918191 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137276c8-7e16-4d2e-8b8a-56ad1b40d110 service placement] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:23:57.918966 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137276c8-7e16-4d2e-8b8a-56ad1b40d110 service placement] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:23:57.919483 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137276c8-7e16-4d2e-8b8a-56ad1b40d110 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:23:57.923090 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-137276c8-7e16-4d2e-8b8a-56ad1b40d110 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:23:57.947630 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137276c8-7e16-4d2e-8b8a-56ad1b40d110 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:23:57.950830 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137276c8-7e16-4d2e-8b8a-56ad1b40d110 service placement] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:23:57.970917 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1121/2256] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:23:57 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:24:07.962614 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=41 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:24:17.163257 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40463067-5956-4dcf-9d34-c82c1350d77b None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:24:17.170070 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40463067-5956-4dcf-9d34-c82c1350d77b None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:24:17.170070 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40463067-5956-4dcf-9d34-c82c1350d77b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:24:17.195646 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-40463067-5956-4dcf-9d34-c82c1350d77b None None] MFA Rules not processed for user `04f64813f09348eb91cb7bb2869f7612`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:24:17.197747 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-40463067-5956-4dcf-9d34-c82c1350d77b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:24:17.210388 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1136/2257] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Jun 2 19:24:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:24:17.223416 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137276c8-7e16-4d2e-8b8a-56ad1b40d110 service placement] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:24:17.235597 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137276c8-7e16-4d2e-8b8a-56ad1b40d110 service placement] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:24:17.241046 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea509a4-3e8f-4ec0-9197-de6c7c977fca service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a59e1d67ff94186a3ebcfcb49938a87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:24:17.242247 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea509a4-3e8f-4ec0-9197-de6c7c977fca service swift] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:24:17.242533 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea509a4-3e8f-4ec0-9197-de6c7c977fca service swift] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:24:17.242849 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea509a4-3e8f-4ec0-9197-de6c7c977fca service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:24:17.252731 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea509a4-3e8f-4ec0-9197-de6c7c977fca service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:24:17.254357 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea509a4-3e8f-4ec0-9197-de6c7c977fca service swift] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:24:17.264130 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1122/2258] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 2 19:24:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 02 19:24:27.186480 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:24:27.259904 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=6 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:25:15.812602 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40463067-5956-4dcf-9d34-c82c1350d77b None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:25:15.816083 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-40463067-5956-4dcf-9d34-c82c1350d77b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:25:15.926833 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40463067-5956-4dcf-9d34-c82c1350d77b None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:25:15.931822 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646f0553-3058-4a02-8218-edf83f983abc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:25:15.932635 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f0553-3058-4a02-8218-edf83f983abc service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:25:15.932962 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f0553-3058-4a02-8218-edf83f983abc service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:25:15.933330 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f0553-3058-4a02-8218-edf83f983abc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:25:15.945309 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646f0553-3058-4a02-8218-edf83f983abc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:25:15.947420 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646f0553-3058-4a02-8218-edf83f983abc service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:25:15.964355 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1137/2259] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:25:15 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:25:22.959401 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a04c5-6366-4e2f-a9a4-c3033a8c4c48 None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:25:22.959914 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a04c5-6366-4e2f-a9a4-c3033a8c4c48 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:25:22.960405 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a04c5-6366-4e2f-a9a4-c3033a8c4c48 None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:25:22.963941 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1123/2260] 10.222.0.33 () {54 vars in 921 bytes} [Thu Jun 2 19:25:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:25:22.976531 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d08232-d35a-4b80-aab2-fe3cca3ac17c None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:25:22.977387 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d08232-d35a-4b80-aab2-fe3cca3ac17c None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:25:22.977387 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d08232-d35a-4b80-aab2-fe3cca3ac17c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:25:23.060430 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-31d08232-d35a-4b80-aab2-fe3cca3ac17c None None] MFA Rules not processed for user `70664d5cbdee47ee89c120761939cc75`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:25:23.061908 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-31d08232-d35a-4b80-aab2-fe3cca3ac17c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:25:23.186021 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1138/2261] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Jun 2 19:25:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:25:23.200233 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a04c5-6366-4e2f-a9a4-c3033a8c4c48 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:25:23.234345 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a04c5-6366-4e2f-a9a4-c3033a8c4c48 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:25:23.240609 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e343ea43-6ff7-47f1-981d-386d7f662347 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07892c0aa21449b94a1cc78e3b8a653', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:25:23.241559 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e343ea43-6ff7-47f1-981d-386d7f662347 service glance] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:25:23.241985 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e343ea43-6ff7-47f1-981d-386d7f662347 service glance] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:25:23.242416 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e343ea43-6ff7-47f1-981d-386d7f662347 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:25:23.262780 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e343ea43-6ff7-47f1-981d-386d7f662347 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:25:23.269453 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e343ea43-6ff7-47f1-981d-386d7f662347 service glance] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:25:23.286530 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1124/2262] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:25:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 02 19:25:23.540892 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485bb228-170b-489f-8bb3-8181a6702708 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:25:23.541415 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485bb228-170b-489f-8bb3-8181a6702708 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:25:23.541631 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485bb228-170b-489f-8bb3-8181a6702708 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:25:23.543778 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1139/2263] 10.222.0.33 () {54 vars in 921 bytes} [Thu Jun 2 19:25:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 02 19:25:23.550545 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6409e331-b7d0-450a-b2ab-ff11e5a3e3ab None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:25:23.550875 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6409e331-b7d0-450a-b2ab-ff11e5a3e3ab None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:25:23.551166 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6409e331-b7d0-450a-b2ab-ff11e5a3e3ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:25:23.581651 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-6409e331-b7d0-450a-b2ab-ff11e5a3e3ab None None] MFA Rules not processed for user `7417a54046e34e899d7b03f2f2314deb`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:25:23.581651 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-6409e331-b7d0-450a-b2ab-ff11e5a3e3ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:25:23.584969 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1125/2264] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Jun 2 19:25:23 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:25:23.601238 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485bb228-170b-489f-8bb3-8181a6702708 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:25:23.603716 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-485bb228-170b-489f-8bb3-8181a6702708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:25:23.621261 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485bb228-170b-489f-8bb3-8181a6702708 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:25:23.625234 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f557d29a-6fa7-494b-8ccd-d85f18051115 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:25:23.625912 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f557d29a-6fa7-494b-8ccd-d85f18051115 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:25:23.626203 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f557d29a-6fa7-494b-8ccd-d85f18051115 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:25:23.626483 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f557d29a-6fa7-494b-8ccd-d85f18051115 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:25:23.636002 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f557d29a-6fa7-494b-8ccd-d85f18051115 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:25:23.637496 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f557d29a-6fa7-494b-8ccd-d85f18051115 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:25:23.646915 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1140/2265] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:25:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 02 19:25:24.891540 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d0fc1-4df2-4aec-90fb-c90bbb795e7f None None] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:25:24.892016 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d0fc1-4df2-4aec-90fb-c90bbb795e7f None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:25:24.892110 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d0fc1-4df2-4aec-90fb-c90bbb795e7f None None] PATH_INFO: `/` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:25:24.894138 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1126/2266] 10.222.0.33 () {54 vars in 921 bytes} [Thu Jun 2 19:25:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:25:24.906127 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f2003-4d08-44a9-9b05-793915a1450c None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:25:24.906686 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f2003-4d08-44a9-9b05-793915a1450c None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:25:24.906847 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f2003-4d08-44a9-9b05-793915a1450c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:25:24.944797 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-994f2003-4d08-44a9-9b05-793915a1450c None None] MFA Rules not processed for user `70664d5cbdee47ee89c120761939cc75`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:25:24.947959 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-994f2003-4d08-44a9-9b05-793915a1450c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:25:24.959205 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1141/2267] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Jun 2 19:25:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:25:24.981163 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d0fc1-4df2-4aec-90fb-c90bbb795e7f None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:25:24.995299 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d0fc1-4df2-4aec-90fb-c90bbb795e7f None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:25:25.000130 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d76d9d-42cc-40bf-b68c-27e64e0811a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:25:25.000797 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d76d9d-42cc-40bf-b68c-27e64e0811a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:25:25.001080 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d76d9d-42cc-40bf-b68c-27e64e0811a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:25:25.001359 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d76d9d-42cc-40bf-b68c-27e64e0811a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:25:25.014656 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d76d9d-42cc-40bf-b68c-27e64e0811a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:25:25.016741 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d76d9d-42cc-40bf-b68c-27e64e0811a3 service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:25:25.032585 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1127/2268] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:25:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:25:34.929626 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=81 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:25:35.026257 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=19 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:25:42.651136 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994f2003-4d08-44a9-9b05-793915a1450c None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:25:42.661052 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994f2003-4d08-44a9-9b05-793915a1450c None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:25:42.664937 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e10176-8746-48f8-9326-30bdc7c40ee2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:25:42.668690 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e10176-8746-48f8-9326-30bdc7c40ee2 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:25:42.668690 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e10176-8746-48f8-9326-30bdc7c40ee2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:25:42.668690 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e10176-8746-48f8-9326-30bdc7c40ee2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:25:42.668690 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-63e10176-8746-48f8-9326-30bdc7c40ee2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:25:42.682862 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e10176-8746-48f8-9326-30bdc7c40ee2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:25:42.684535 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e10176-8746-48f8-9326-30bdc7c40ee2 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:25:42.697775 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1142/2269] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:25:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:25:45.082558 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d76d9d-42cc-40bf-b68c-27e64e0811a3 service neutron] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:25:45.091728 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d76d9d-42cc-40bf-b68c-27e64e0811a3 service neutron] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:25:45.095562 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708f77b8-caa1-493c-8fee-147fed815190 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:25:45.096383 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708f77b8-caa1-493c-8fee-147fed815190 service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:25:45.096507 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708f77b8-caa1-493c-8fee-147fed815190 service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:25:45.096796 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708f77b8-caa1-493c-8fee-147fed815190 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:25:45.098729 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-708f77b8-caa1-493c-8fee-147fed815190 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:25:45.128339 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708f77b8-caa1-493c-8fee-147fed815190 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:25:45.129808 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708f77b8-caa1-493c-8fee-147fed815190 service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:25:45.143686 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1128/2270] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:25:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:25:52.691557 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=6 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:25:55.137441 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=6 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:25:57.342742 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e10176-8746-48f8-9326-30bdc7c40ee2 service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:25:57.344221 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-63e10176-8746-48f8-9326-30bdc7c40ee2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:25:57.427167 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e10176-8746-48f8-9326-30bdc7c40ee2 service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:25:57.430805 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e838fbb3-5dd1-4f04-b489-4f6d4e503bc4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:25:57.431463 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e838fbb3-5dd1-4f04-b489-4f6d4e503bc4 service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:25:57.431750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e838fbb3-5dd1-4f04-b489-4f6d4e503bc4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:25:57.432028 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e838fbb3-5dd1-4f04-b489-4f6d4e503bc4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:25:57.434463 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-e838fbb3-5dd1-4f04-b489-4f6d4e503bc4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:25:57.503474 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e838fbb3-5dd1-4f04-b489-4f6d4e503bc4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:25:57.504965 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e838fbb3-5dd1-4f04-b489-4f6d4e503bc4 service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:25:57.516949 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1143/2271] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:25:57 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:26:00.954303 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708f77b8-caa1-493c-8fee-147fed815190 service neutron] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:26:00.955842 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-708f77b8-caa1-493c-8fee-147fed815190 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:26:00.974564 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708f77b8-caa1-493c-8fee-147fed815190 service neutron] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:26:00.977229 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2613b18c-39e7-43dc-a0b1-ddc8d423f52d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:26:00.977930 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2613b18c-39e7-43dc-a0b1-ddc8d423f52d service nova] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:26:00.978244 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2613b18c-39e7-43dc-a0b1-ddc8d423f52d service nova] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:26:00.978544 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2613b18c-39e7-43dc-a0b1-ddc8d423f52d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:26:00.980902 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-2613b18c-39e7-43dc-a0b1-ddc8d423f52d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:26:01.046120 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2613b18c-39e7-43dc-a0b1-ddc8d423f52d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:26:01.047743 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2613b18c-39e7-43dc-a0b1-ddc8d423f52d service nova] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:26:01.057200 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1129/2272] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:26:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:26:02.409962 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e838fbb3-5dd1-4f04-b489-4f6d4e503bc4 service cinder] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:26:02.419486 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e838fbb3-5dd1-4f04-b489-4f6d4e503bc4 service cinder] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:26:02.423384 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd4aece-69f6-4ad3-9c1c-0eb675994696 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:26:02.424086 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd4aece-69f6-4ad3-9c1c-0eb675994696 service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:26:02.424425 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd4aece-69f6-4ad3-9c1c-0eb675994696 service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:26:02.424702 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd4aece-69f6-4ad3-9c1c-0eb675994696 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:26:02.426704 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-7cd4aece-69f6-4ad3-9c1c-0eb675994696 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:26:02.489978 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd4aece-69f6-4ad3-9c1c-0eb675994696 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:26:02.491527 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd4aece-69f6-4ad3-9c1c-0eb675994696 service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:26:02.502313 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1144/2273] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:26:02 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:26:11.052689 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=28 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:26:12.497847 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=85 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:26:18.504103 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2613b18c-39e7-43dc-a0b1-ddc8d423f52d service nova] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:26:18.518130 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2613b18c-39e7-43dc-a0b1-ddc8d423f52d service nova] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:26:18.528120 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6d929c-c4b1-4496-960a-cf63bb949aac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:26:18.528847 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d929c-c4b1-4496-960a-cf63bb949aac service nova] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:26:18.529170 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d929c-c4b1-4496-960a-cf63bb949aac service nova] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:26:18.529471 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d929c-c4b1-4496-960a-cf63bb949aac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:26:18.531458 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-1d6d929c-c4b1-4496-960a-cf63bb949aac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:26:18.549749 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6d929c-c4b1-4496-960a-cf63bb949aac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:26:18.551460 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6d929c-c4b1-4496-960a-cf63bb949aac service nova] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:26:18.594038 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1130/2274] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:26:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:26:18.672254 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd4aece-69f6-4ad3-9c1c-0eb675994696 service cinder] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:26:18.684465 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd4aece-69f6-4ad3-9c1c-0eb675994696 service cinder] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:26:18.689504 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5390ad-eb6a-41cd-9c0b-c708f6da4669 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:26:18.690449 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5390ad-eb6a-41cd-9c0b-c708f6da4669 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:26:18.690930 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5390ad-eb6a-41cd-9c0b-c708f6da4669 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:26:18.691396 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5390ad-eb6a-41cd-9c0b-c708f6da4669 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:26:18.693636 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-ce5390ad-eb6a-41cd-9c0b-c708f6da4669 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:26:18.715200 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5390ad-eb6a-41cd-9c0b-c708f6da4669 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:26:18.716767 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5390ad-eb6a-41cd-9c0b-c708f6da4669 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:26:18.733142 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1145/2275] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:26:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:26:28.570689 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=6 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:26:28.722734 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=6 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:27:24.603956 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6d929c-c4b1-4496-960a-cf63bb949aac service nova] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:27:24.605382 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-1d6d929c-c4b1-4496-960a-cf63bb949aac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:27:24.619105 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6d929c-c4b1-4496-960a-cf63bb949aac service nova] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:27:24.624821 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6595a1e6-4f17-4fd5-8a62-c71a6e4463c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:27:24.625460 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6595a1e6-4f17-4fd5-8a62-c71a6e4463c7 service nova] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:27:24.625748 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6595a1e6-4f17-4fd5-8a62-c71a6e4463c7 service nova] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:27:24.626025 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6595a1e6-4f17-4fd5-8a62-c71a6e4463c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:27:24.627793 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-6595a1e6-4f17-4fd5-8a62-c71a6e4463c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:27:24.686231 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6595a1e6-4f17-4fd5-8a62-c71a6e4463c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:27:24.687789 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6595a1e6-4f17-4fd5-8a62-c71a6e4463c7 service nova] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:27:24.697391 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1131/2276] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:27:24 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:27:34.693206 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=28 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:27:48.309912 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5390ad-eb6a-41cd-9c0b-c708f6da4669 service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:27:48.311615 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-ce5390ad-eb6a-41cd-9c0b-c708f6da4669 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:27:48.323108 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5390ad-eb6a-41cd-9c0b-c708f6da4669 service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:27:48.326695 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2128394a-9a42-462c-ba77-0c19fc7a890e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:27:48.327389 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2128394a-9a42-462c-ba77-0c19fc7a890e service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:27:48.327705 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2128394a-9a42-462c-ba77-0c19fc7a890e service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:27:48.327985 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2128394a-9a42-462c-ba77-0c19fc7a890e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:27:48.329792 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-2128394a-9a42-462c-ba77-0c19fc7a890e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:27:48.390283 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2128394a-9a42-462c-ba77-0c19fc7a890e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:27:48.391779 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2128394a-9a42-462c-ba77-0c19fc7a890e service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:27:48.401287 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1146/2277] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:27:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:27:58.397120 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=28 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:28:23.388257 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6595a1e6-4f17-4fd5-8a62-c71a6e4463c7 service nova] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:28:23.399355 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6595a1e6-4f17-4fd5-8a62-c71a6e4463c7 service nova] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:28:23.404744 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97fc711-b8f3-48df-a535-6f250a478c64 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:28:23.405395 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97fc711-b8f3-48df-a535-6f250a478c64 service nova] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:28:23.405684 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97fc711-b8f3-48df-a535-6f250a478c64 service nova] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:28:23.405963 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97fc711-b8f3-48df-a535-6f250a478c64 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:28:23.414161 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97fc711-b8f3-48df-a535-6f250a478c64 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:28:23.416374 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97fc711-b8f3-48df-a535-6f250a478c64 service nova] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:28:23.433145 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1132/2278] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:28:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:28:33.420857 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=6 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:29:47.824162 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2128394a-9a42-462c-ba77-0c19fc7a890e service cinder] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:29:47.838261 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2128394a-9a42-462c-ba77-0c19fc7a890e service cinder] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:29:47.845605 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6165a9-8255-41b8-ae5d-df5ef7691a09 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73ae837e99764bebbe7a6e06ac9049e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:29:47.847039 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6165a9-8255-41b8-ae5d-df5ef7691a09 service placement] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:29:47.847039 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6165a9-8255-41b8-ae5d-df5ef7691a09 service placement] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:29:47.847344 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6165a9-8255-41b8-ae5d-df5ef7691a09 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:29:47.861778 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6165a9-8255-41b8-ae5d-df5ef7691a09 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:29:47.863450 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6165a9-8255-41b8-ae5d-df5ef7691a09 service placement] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:29:47.875559 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1147/2279] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:29:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:29:57.871301 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=6 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:30:42.769556 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97fc711-b8f3-48df-a535-6f250a478c64 service nova] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:30:42.779570 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97fc711-b8f3-48df-a535-6f250a478c64 service nova] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:30:42.783565 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf54c66-e414-41c8-9954-5fadcfa00961 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:30:42.784226 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf54c66-e414-41c8-9954-5fadcfa00961 service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:30:42.784511 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf54c66-e414-41c8-9954-5fadcfa00961 service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:30:42.784791 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf54c66-e414-41c8-9954-5fadcfa00961 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:30:42.796095 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf54c66-e414-41c8-9954-5fadcfa00961 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:30:42.797544 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf54c66-e414-41c8-9954-5fadcfa00961 service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:30:42.809426 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1133/2280] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:30:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:30:46.393032 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6165a9-8255-41b8-ae5d-df5ef7691a09 service placement] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:30:46.402908 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6165a9-8255-41b8-ae5d-df5ef7691a09 service placement] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:30:46.406521 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a1ccc8-b892-4c48-8bef-bfa803fa3441 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:30:46.407203 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a1ccc8-b892-4c48-8bef-bfa803fa3441 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:30:46.407495 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a1ccc8-b892-4c48-8bef-bfa803fa3441 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:30:46.407772 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a1ccc8-b892-4c48-8bef-bfa803fa3441 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:30:46.419078 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a1ccc8-b892-4c48-8bef-bfa803fa3441 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:30:46.420567 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a1ccc8-b892-4c48-8bef-bfa803fa3441 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:30:46.432436 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1148/2281] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:30:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:30:52.804087 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=6 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:30:56.427511 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=6 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:30:57.787928 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf54c66-e414-41c8-9954-5fadcfa00961 service neutron] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:30:57.797589 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf54c66-e414-41c8-9954-5fadcfa00961 service neutron] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:30:57.802019 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a52e047-0827-4a51-a7cc-0e0033a0b374 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:30:57.802697 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a52e047-0827-4a51-a7cc-0e0033a0b374 service cinder] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:30:57.803014 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a52e047-0827-4a51-a7cc-0e0033a0b374 service cinder] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:30:57.803296 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a52e047-0827-4a51-a7cc-0e0033a0b374 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:30:57.812275 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a52e047-0827-4a51-a7cc-0e0033a0b374 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:30:57.813721 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a52e047-0827-4a51-a7cc-0e0033a0b374 service cinder] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:30:57.841609 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1134/2282] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:30:57 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:31:00.973131 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a1ccc8-b892-4c48-8bef-bfa803fa3441 service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:31:00.985002 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a1ccc8-b892-4c48-8bef-bfa803fa3441 service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:31:00.991204 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7306d438-442d-4b18-8d89-3c1ad5106f43 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:31:00.991872 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306d438-442d-4b18-8d89-3c1ad5106f43 service nova] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:31:00.992160 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306d438-442d-4b18-8d89-3c1ad5106f43 service nova] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:31:00.992435 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306d438-442d-4b18-8d89-3c1ad5106f43 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:31:01.001836 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7306d438-442d-4b18-8d89-3c1ad5106f43 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:31:01.003293 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7306d438-442d-4b18-8d89-3c1ad5106f43 service nova] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:31:01.034545 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1149/2283] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:31:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:31:03.184312 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a52e047-0827-4a51-a7cc-0e0033a0b374 service cinder] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:31:03.200905 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a52e047-0827-4a51-a7cc-0e0033a0b374 service cinder] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:31:03.206458 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404c99ca-ef1b-4a18-8e79-75c9ddc438a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:31:03.207276 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404c99ca-ef1b-4a18-8e79-75c9ddc438a9 service nova] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:31:03.208246 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404c99ca-ef1b-4a18-8e79-75c9ddc438a9 service nova] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:31:03.208439 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404c99ca-ef1b-4a18-8e79-75c9ddc438a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:31:03.217048 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404c99ca-ef1b-4a18-8e79-75c9ddc438a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:31:03.218647 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404c99ca-ef1b-4a18-8e79-75c9ddc438a9 service nova] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:31:03.246436 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1135/2284] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:31:03 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 02 19:31:11.031568 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=13 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:31:13.243539 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=26 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:31:54.461228 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7306d438-442d-4b18-8d89-3c1ad5106f43 service nova] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:31:54.471709 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7306d438-442d-4b18-8d89-3c1ad5106f43 service nova] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:31:54.475339 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dbe34f-fdb8-4181-89ba-6fc6395e6177 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:31:54.475982 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dbe34f-fdb8-4181-89ba-6fc6395e6177 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:31:54.476267 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dbe34f-fdb8-4181-89ba-6fc6395e6177 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:31:54.476547 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dbe34f-fdb8-4181-89ba-6fc6395e6177 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:31:54.484922 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dbe34f-fdb8-4181-89ba-6fc6395e6177 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:31:54.486388 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dbe34f-fdb8-4181-89ba-6fc6395e6177 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:31:54.494331 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1150/2285] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:31:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:32:04.490879 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=6 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:32:24.539239 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404c99ca-ef1b-4a18-8e79-75c9ddc438a9 service nova] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:32:24.554474 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404c99ca-ef1b-4a18-8e79-75c9ddc438a9 service nova] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:32:24.560773 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1788a00c-7007-421e-8ee7-5a8731303d93 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:32:24.561442 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788a00c-7007-421e-8ee7-5a8731303d93 service nova] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:32:24.561733 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788a00c-7007-421e-8ee7-5a8731303d93 service nova] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:32:24.562016 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788a00c-7007-421e-8ee7-5a8731303d93 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:32:24.576937 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1788a00c-7007-421e-8ee7-5a8731303d93 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:32:24.578584 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1788a00c-7007-421e-8ee7-5a8731303d93 service nova] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:32:24.607813 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1136/2286] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:32:24 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:32:34.604839 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=13 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:32:49.383536 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dbe34f-fdb8-4181-89ba-6fc6395e6177 service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:32:49.403959 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dbe34f-fdb8-4181-89ba-6fc6395e6177 service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:32:49.407683 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6044ec69-81f4-4328-b4b4-9a88afe153f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:32:49.408334 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6044ec69-81f4-4328-b4b4-9a88afe153f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:32:49.408620 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6044ec69-81f4-4328-b4b4-9a88afe153f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:32:49.408898 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6044ec69-81f4-4328-b4b4-9a88afe153f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:32:49.434251 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6044ec69-81f4-4328-b4b4-9a88afe153f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:32:49.436655 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6044ec69-81f4-4328-b4b4-9a88afe153f6 service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:32:49.469572 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1151/2287] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:32:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:32:59.464462 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=24 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:33:23.679333 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1788a00c-7007-421e-8ee7-5a8731303d93 service nova] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:33:23.698969 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1788a00c-7007-421e-8ee7-5a8731303d93 service nova] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:33:23.702514 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9771afea-16b2-4b0d-9b2c-370320f5c9e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:33:23.703206 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9771afea-16b2-4b0d-9b2c-370320f5c9e2 service cinder] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:33:23.703542 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9771afea-16b2-4b0d-9b2c-370320f5c9e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:33:23.703819 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9771afea-16b2-4b0d-9b2c-370320f5c9e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:33:23.705803 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-9771afea-16b2-4b0d-9b2c-370320f5c9e2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:33:23.801746 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9771afea-16b2-4b0d-9b2c-370320f5c9e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:33:23.803322 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9771afea-16b2-4b0d-9b2c-370320f5c9e2 service cinder] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:33:23.823639 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1137/2288] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:33:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:33:33.820179 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=50 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:34:50.742358 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6044ec69-81f4-4328-b4b4-9a88afe153f6 service cinder] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:34:50.744280 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-6044ec69-81f4-4328-b4b4-9a88afe153f6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:34:50.853112 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6044ec69-81f4-4328-b4b4-9a88afe153f6 service cinder] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:34:50.858219 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9239e18f-9012-41ed-9c08-c07efe9427ba service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73ae837e99764bebbe7a6e06ac9049e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:34:50.858913 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9239e18f-9012-41ed-9c08-c07efe9427ba service placement] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:34:50.859201 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9239e18f-9012-41ed-9c08-c07efe9427ba service placement] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:34:50.859481 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9239e18f-9012-41ed-9c08-c07efe9427ba service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:34:50.861417 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-9239e18f-9012-41ed-9c08-c07efe9427ba service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:34:50.873845 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9239e18f-9012-41ed-9c08-c07efe9427ba service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:34:50.875483 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9239e18f-9012-41ed-9c08-c07efe9427ba service placement] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:34:50.885618 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1152/2289] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:34:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:35:00.881404 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=48 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:35:56.704294 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9771afea-16b2-4b0d-9b2c-370320f5c9e2 service cinder] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:35:56.712925 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9771afea-16b2-4b0d-9b2c-370320f5c9e2 service cinder] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:35:56.716841 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65162a6e-a56f-4582-8d9c-0eeb3f66d4d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:35:56.717552 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65162a6e-a56f-4582-8d9c-0eeb3f66d4d8 service cinder] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:35:56.717841 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65162a6e-a56f-4582-8d9c-0eeb3f66d4d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:35:56.718145 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65162a6e-a56f-4582-8d9c-0eeb3f66d4d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:35:56.727207 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65162a6e-a56f-4582-8d9c-0eeb3f66d4d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:35:56.728635 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65162a6e-a56f-4582-8d9c-0eeb3f66d4d8 service cinder] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:35:56.737711 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1138/2290] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:35:56 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:36:00.063633 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9239e18f-9012-41ed-9c08-c07efe9427ba service placement] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:36:00.132931 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9239e18f-9012-41ed-9c08-c07efe9427ba service placement] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:36:00.136587 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868a9abc-f8ca-4d5f-870f-e5733cc110cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:36:00.137241 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868a9abc-f8ca-4d5f-870f-e5733cc110cd service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:36:00.137528 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868a9abc-f8ca-4d5f-870f-e5733cc110cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:36:00.137806 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868a9abc-f8ca-4d5f-870f-e5733cc110cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:36:00.147620 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868a9abc-f8ca-4d5f-870f-e5733cc110cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:36:00.149347 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868a9abc-f8ca-4d5f-870f-e5733cc110cd service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:36:00.159741 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1153/2291] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:36:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:36:02.684681 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65162a6e-a56f-4582-8d9c-0eeb3f66d4d8 service cinder] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:36:02.686150 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-65162a6e-a56f-4582-8d9c-0eeb3f66d4d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:36:02.789690 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65162a6e-a56f-4582-8d9c-0eeb3f66d4d8 service cinder] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:36:02.795460 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60d4d0e-c0fc-4051-9602-7bca232edc44 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:36:02.796112 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d4d0e-c0fc-4051-9602-7bca232edc44 service nova] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:36:02.796400 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d4d0e-c0fc-4051-9602-7bca232edc44 service nova] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:36:02.796678 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d4d0e-c0fc-4051-9602-7bca232edc44 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:36:02.798604 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-b60d4d0e-c0fc-4051-9602-7bca232edc44 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:36:02.871770 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60d4d0e-c0fc-4051-9602-7bca232edc44 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:36:02.873398 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60d4d0e-c0fc-4051-9602-7bca232edc44 service nova] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:36:02.883182 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1139/2292] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:36:02 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:36:10.154646 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=30 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:36:12.878765 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=79 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:36:14.013932 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868a9abc-f8ca-4d5f-870f-e5733cc110cd service cinder] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:36:14.015648 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-868a9abc-f8ca-4d5f-870f-e5733cc110cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:36:14.093041 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868a9abc-f8ca-4d5f-870f-e5733cc110cd service cinder] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:36:14.096674 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21f1bd0-54cd-4dcb-b26e-38c841cf15fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:36:14.097260 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f1bd0-54cd-4dcb-b26e-38c841cf15fd service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:36:14.097544 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f1bd0-54cd-4dcb-b26e-38c841cf15fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:36:14.097815 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f1bd0-54cd-4dcb-b26e-38c841cf15fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:36:14.099729 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-b21f1bd0-54cd-4dcb-b26e-38c841cf15fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:36:14.112673 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21f1bd0-54cd-4dcb-b26e-38c841cf15fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:36:14.114169 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21f1bd0-54cd-4dcb-b26e-38c841cf15fd service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:36:14.125342 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1154/2293] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:36:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:36:15.299288 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60d4d0e-c0fc-4051-9602-7bca232edc44 service nova] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:36:15.308515 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60d4d0e-c0fc-4051-9602-7bca232edc44 service nova] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:36:15.312114 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c802e71f-7db4-474f-82e5-4846a7d5ab1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:36:15.312754 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c802e71f-7db4-474f-82e5-4846a7d5ab1e service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:36:15.313038 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c802e71f-7db4-474f-82e5-4846a7d5ab1e service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:36:15.313316 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c802e71f-7db4-474f-82e5-4846a7d5ab1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:36:15.315226 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-c802e71f-7db4-474f-82e5-4846a7d5ab1e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:36:15.328143 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c802e71f-7db4-474f-82e5-4846a7d5ab1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:36:15.329558 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c802e71f-7db4-474f-82e5-4846a7d5ab1e service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:36:15.340863 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1140/2294] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:36:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:36:24.120446 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=35 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:36:25.336045 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=6 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:37:01.510328 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21f1bd0-54cd-4dcb-b26e-38c841cf15fd service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:37:01.518843 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21f1bd0-54cd-4dcb-b26e-38c841cf15fd service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:37:01.523086 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7179cf-a407-43b3-8ab1-9a8831b3c6b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:37:01.523724 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7179cf-a407-43b3-8ab1-9a8831b3c6b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:37:01.524013 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7179cf-a407-43b3-8ab1-9a8831b3c6b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:37:01.524288 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7179cf-a407-43b3-8ab1-9a8831b3c6b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:37:01.526173 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-7c7179cf-a407-43b3-8ab1-9a8831b3c6b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:37:01.536996 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7179cf-a407-43b3-8ab1-9a8831b3c6b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:37:01.538436 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7179cf-a407-43b3-8ab1-9a8831b3c6b6 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:37:01.546832 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1155/2295] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:37:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:37:11.542963 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=6 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:37:23.917391 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c802e71f-7db4-474f-82e5-4846a7d5ab1e service neutron] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:37:23.930888 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c802e71f-7db4-474f-82e5-4846a7d5ab1e service neutron] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:37:23.936486 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d310ba18-1f1d-474e-ba38-5bfa47cf44a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:37:23.937138 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d310ba18-1f1d-474e-ba38-5bfa47cf44a1 service nova] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:37:23.937420 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d310ba18-1f1d-474e-ba38-5bfa47cf44a1 service nova] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:37:23.937693 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d310ba18-1f1d-474e-ba38-5bfa47cf44a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:37:23.947075 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d310ba18-1f1d-474e-ba38-5bfa47cf44a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:37:23.948691 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d310ba18-1f1d-474e-ba38-5bfa47cf44a1 service nova] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:37:23.957750 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1141/2296] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:37:23 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:37:33.953414 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=6 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:37:47.832224 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7179cf-a407-43b3-8ab1-9a8831b3c6b6 service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:37:47.841279 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7179cf-a407-43b3-8ab1-9a8831b3c6b6 service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:37:47.845026 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0404ffb2-272c-4c60-ae9a-50a099aca309 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:37:47.845675 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0404ffb2-272c-4c60-ae9a-50a099aca309 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:37:47.845970 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0404ffb2-272c-4c60-ae9a-50a099aca309 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:37:47.846252 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0404ffb2-272c-4c60-ae9a-50a099aca309 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:37:47.855611 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0404ffb2-272c-4c60-ae9a-50a099aca309 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:37:47.857045 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0404ffb2-272c-4c60-ae9a-50a099aca309 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:37:47.866296 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1156/2297] 10.222.0.33 () {60 vars in 1396 bytes} [Thu Jun 2 19:37:47 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3250 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:37:57.861903 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=6 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:38:23.865048 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d310ba18-1f1d-474e-ba38-5bfa47cf44a1 service nova] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:38:23.876838 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d310ba18-1f1d-474e-ba38-5bfa47cf44a1 service nova] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:38:23.882478 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0deb3c-2149-4b9c-863c-c00805888deb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:38:23.883155 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0deb3c-2149-4b9c-863c-c00805888deb service nova] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:38:23.883446 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0deb3c-2149-4b9c-863c-c00805888deb service nova] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:38:23.884768 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0deb3c-2149-4b9c-863c-c00805888deb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:38:23.892655 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0deb3c-2149-4b9c-863c-c00805888deb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:38:23.894238 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0deb3c-2149-4b9c-863c-c00805888deb service nova] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:38:23.903193 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1142/2298] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:38:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:38:33.899059 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=6 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:39:53.795869 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0404ffb2-272c-4c60-ae9a-50a099aca309 service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:39:53.798261 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-0404ffb2-272c-4c60-ae9a-50a099aca309 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:39:53.814273 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0404ffb2-272c-4c60-ae9a-50a099aca309 service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:39:53.819442 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4433f02c-f40f-483f-a3c1-74cd81844b41 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73ae837e99764bebbe7a6e06ac9049e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:39:53.820092 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4433f02c-f40f-483f-a3c1-74cd81844b41 service placement] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:39:53.820383 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4433f02c-f40f-483f-a3c1-74cd81844b41 service placement] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:39:53.820666 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4433f02c-f40f-483f-a3c1-74cd81844b41 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:39:53.830208 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4433f02c-f40f-483f-a3c1-74cd81844b41 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:39:53.831850 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4433f02c-f40f-483f-a3c1-74cd81844b41 service placement] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:39:53.842008 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1157/2299] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:39:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:40:03.838090 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=6 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:40:57.007569 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0deb3c-2149-4b9c-863c-c00805888deb service nova] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:40:57.009128 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-ae0deb3c-2149-4b9c-863c-c00805888deb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:40:57.033834 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0deb3c-2149-4b9c-863c-c00805888deb service nova] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:40:57.037442 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e92c18-2709-441e-8f74-71440d78a63c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:40:57.038087 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e92c18-2709-441e-8f74-71440d78a63c service cinder] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:40:57.038431 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e92c18-2709-441e-8f74-71440d78a63c service cinder] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:40:57.038661 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e92c18-2709-441e-8f74-71440d78a63c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:40:57.040655 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-46e92c18-2709-441e-8f74-71440d78a63c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:40:57.109202 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e92c18-2709-441e-8f74-71440d78a63c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:40:57.111232 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e92c18-2709-441e-8f74-71440d78a63c service cinder] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:40:57.122352 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1143/2300] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:40:57 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:41:00.031992 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4433f02c-f40f-483f-a3c1-74cd81844b41 service placement] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:41:00.042657 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4433f02c-f40f-483f-a3c1-74cd81844b41 service placement] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:41:00.046316 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6506dad0-22c6-4dd3-92a5-c28a7db062f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:41:00.047011 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6506dad0-22c6-4dd3-92a5-c28a7db062f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:41:00.047303 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6506dad0-22c6-4dd3-92a5-c28a7db062f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:41:00.047585 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6506dad0-22c6-4dd3-92a5-c28a7db062f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:41:00.049799 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-6506dad0-22c6-4dd3-92a5-c28a7db062f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:41:00.141039 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6506dad0-22c6-4dd3-92a5-c28a7db062f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:41:00.143404 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6506dad0-22c6-4dd3-92a5-c28a7db062f3 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:41:00.153931 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1158/2301] 10.222.0.33 () {60 vars in 1396 bytes} [Thu Jun 2 19:41:00 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3252 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:41:03.221531 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e92c18-2709-441e-8f74-71440d78a63c service cinder] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:41:03.230047 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e92c18-2709-441e-8f74-71440d78a63c service cinder] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:41:03.233706 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fe9840-b0ca-4c23-bc09-13a2a0296b82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:41:03.234343 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe9840-b0ca-4c23-bc09-13a2a0296b82 service cinder] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:41:03.234632 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe9840-b0ca-4c23-bc09-13a2a0296b82 service cinder] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:41:03.234945 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe9840-b0ca-4c23-bc09-13a2a0296b82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:41:03.243342 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fe9840-b0ca-4c23-bc09-13a2a0296b82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:41:03.244769 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fe9840-b0ca-4c23-bc09-13a2a0296b82 service cinder] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:41:03.272801 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1144/2302] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:41:03 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:41:10.149932 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=33 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:41:13.270092 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=51 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:41:15.082591 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6506dad0-22c6-4dd3-92a5-c28a7db062f3 service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:41:15.085905 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fe9840-b0ca-4c23-bc09-13a2a0296b82 service cinder] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:41:15.091132 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6506dad0-22c6-4dd3-92a5-c28a7db062f3 service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:41:15.094738 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0873c70f-af76-44fd-930d-795f53208ca3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:41:15.095396 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0873c70f-af76-44fd-930d-795f53208ca3 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:41:15.095676 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0873c70f-af76-44fd-930d-795f53208ca3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:41:15.095798 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fe9840-b0ca-4c23-bc09-13a2a0296b82 service cinder] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:41:15.095949 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0873c70f-af76-44fd-930d-795f53208ca3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:41:15.099461 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3ac3b1-f0bf-4049-b4b9-31705a8c5eee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:41:15.100200 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ac3b1-f0bf-4049-b4b9-31705a8c5eee service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:41:15.100546 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ac3b1-f0bf-4049-b4b9-31705a8c5eee service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:41:15.100886 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ac3b1-f0bf-4049-b4b9-31705a8c5eee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:41:15.106711 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0873c70f-af76-44fd-930d-795f53208ca3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:41:15.108307 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0873c70f-af76-44fd-930d-795f53208ca3 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:41:15.116388 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3ac3b1-f0bf-4049-b4b9-31705a8c5eee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:41:15.116388 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3ac3b1-f0bf-4049-b4b9-31705a8c5eee service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:41:15.120139 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1159/2303] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:41:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:41:15.128789 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1145/2304] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:41:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:41:25.114842 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=6 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:41:25.121425 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=6 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:42:05.485441 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0873c70f-af76-44fd-930d-795f53208ca3 service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:42:05.498329 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0873c70f-af76-44fd-930d-795f53208ca3 service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:42:05.502201 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1286a48-02f1-46ef-8d60-43bfbdaf45c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:42:05.502870 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1286a48-02f1-46ef-8d60-43bfbdaf45c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:42:05.503158 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1286a48-02f1-46ef-8d60-43bfbdaf45c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:42:05.503437 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1286a48-02f1-46ef-8d60-43bfbdaf45c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:42:05.512596 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1286a48-02f1-46ef-8d60-43bfbdaf45c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:42:05.514280 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1286a48-02f1-46ef-8d60-43bfbdaf45c9 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:42:05.522349 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1160/2305] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:42:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:42:15.518658 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=6 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:42:24.203422 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3ac3b1-f0bf-4049-b4b9-31705a8c5eee service neutron] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:42:24.207739 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-1f3ac3b1-f0bf-4049-b4b9-31705a8c5eee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:42:24.220406 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3ac3b1-f0bf-4049-b4b9-31705a8c5eee service neutron] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:42:24.226495 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e99a0e-6c8b-4c70-9da7-39f03dc7d6ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:42:24.227181 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e99a0e-6c8b-4c70-9da7-39f03dc7d6ae service nova] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:42:24.227465 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e99a0e-6c8b-4c70-9da7-39f03dc7d6ae service nova] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:42:24.227738 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e99a0e-6c8b-4c70-9da7-39f03dc7d6ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:42:24.229637 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-b3e99a0e-6c8b-4c70-9da7-39f03dc7d6ae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:42:24.242411 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e99a0e-6c8b-4c70-9da7-39f03dc7d6ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:42:24.244399 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e99a0e-6c8b-4c70-9da7-39f03dc7d6ae service nova] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:42:24.257043 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1146/2306] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:42:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:42:24.772070 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1286a48-02f1-46ef-8d60-43bfbdaf45c9 service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:42:24.773624 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-d1286a48-02f1-46ef-8d60-43bfbdaf45c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:42:24.785310 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1286a48-02f1-46ef-8d60-43bfbdaf45c9 service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:42:24.788891 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b12b2a-0014-4207-b711-7db474b522df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:42:24.789531 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b12b2a-0014-4207-b711-7db474b522df service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:42:24.789812 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b12b2a-0014-4207-b711-7db474b522df service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:42:24.790082 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b12b2a-0014-4207-b711-7db474b522df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:42:24.791944 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-b1b12b2a-0014-4207-b711-7db474b522df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:42:24.871570 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b12b2a-0014-4207-b711-7db474b522df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:42:24.873014 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b12b2a-0014-4207-b711-7db474b522df service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:42:24.891882 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1161/2307] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:42:24 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:42:28.007724 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e99a0e-6c8b-4c70-9da7-39f03dc7d6ae service nova] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:42:28.009327 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-b3e99a0e-6c8b-4c70-9da7-39f03dc7d6ae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:42:28.021406 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e99a0e-6c8b-4c70-9da7-39f03dc7d6ae service nova] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:42:28.024964 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9851422-be2c-4a2b-ac7c-3f7ed3bd6b0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:42:28.025598 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9851422-be2c-4a2b-ac7c-3f7ed3bd6b0f service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:42:28.025886 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9851422-be2c-4a2b-ac7c-3f7ed3bd6b0f service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:42:28.026166 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9851422-be2c-4a2b-ac7c-3f7ed3bd6b0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:42:28.027876 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-d9851422-be2c-4a2b-ac7c-3f7ed3bd6b0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:42:28.120705 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9851422-be2c-4a2b-ac7c-3f7ed3bd6b0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:42:28.121193 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9851422-be2c-4a2b-ac7c-3f7ed3bd6b0f service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:42:28.141112 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1147/2308] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:42:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:42:31.855299 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b12b2a-0014-4207-b711-7db474b522df service cinder] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:42:31.867352 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b12b2a-0014-4207-b711-7db474b522df service cinder] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:42:31.872509 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ab1635-8e1b-4c31-84a7-33fbe0a9e6f2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73ae837e99764bebbe7a6e06ac9049e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:42:31.873164 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ab1635-8e1b-4c31-84a7-33fbe0a9e6f2 service placement] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:42:31.873451 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ab1635-8e1b-4c31-84a7-33fbe0a9e6f2 service placement] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:42:31.873738 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ab1635-8e1b-4c31-84a7-33fbe0a9e6f2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:42:31.875570 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-c8ab1635-8e1b-4c31-84a7-33fbe0a9e6f2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:42:31.888054 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ab1635-8e1b-4c31-84a7-33fbe0a9e6f2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:42:31.889940 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ab1635-8e1b-4c31-84a7-33fbe0a9e6f2 service placement] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:42:31.900938 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1162/2309] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:42:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:42:31.947489 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9851422-be2c-4a2b-ac7c-3f7ed3bd6b0f service neutron] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:42:31.949023 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-d9851422-be2c-4a2b-ac7c-3f7ed3bd6b0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:42:31.962590 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9851422-be2c-4a2b-ac7c-3f7ed3bd6b0f service neutron] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:42:31.967774 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3e6d96-bad3-4b17-820c-b654131bb853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:42:31.968116 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e6d96-bad3-4b17-820c-b654131bb853 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:42:31.968398 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e6d96-bad3-4b17-820c-b654131bb853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:42:31.968672 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e6d96-bad3-4b17-820c-b654131bb853 admin admin] PATH_INFO: `/v3/users/7f950b5cc5fa44c988b945e9cd553874` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:42:31.969869 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3e6d96-bad3-4b17-820c-b654131bb853 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f950b5cc5fa44c988b945e9cd553874)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:42:31.971819 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3e6d96-bad3-4b17-820c-b654131bb853 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:42:32.099805 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-ce3e6d96-bad3-4b17-820c-b654131bb853 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f950b5cc5fa44c988b945e9cd553874'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:42:32.106776 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-ce3e6d96-bad3-4b17-820c-b654131bb853 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f950b5cc5fa44c988b945e9cd553874'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:42:32.107465 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-ce3e6d96-bad3-4b17-820c-b654131bb853 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f950b5cc5fa44c988b945e9cd553874'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:42:32.178681 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-ce3e6d96-bad3-4b17-820c-b654131bb853 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f950b5cc5fa44c988b945e9cd553874'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:42:32.179356 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-ce3e6d96-bad3-4b17-820c-b654131bb853 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f950b5cc5fa44c988b945e9cd553874'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:42:32.197434 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1148/2310] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:42:31 2022] DELETE /identity/v3/users/7f950b5cc5fa44c988b945e9cd553874 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:42:32.460965 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ab1635-8e1b-4c31-84a7-33fbe0a9e6f2 service placement] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:42:32.471499 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ab1635-8e1b-4c31-84a7-33fbe0a9e6f2 service placement] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:42:32.532000 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2825dcdb-f3ab-4a34-8e82-4406d183abf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:42:32.532979 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2825dcdb-f3ab-4a34-8e82-4406d183abf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:42:32.533146 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2825dcdb-f3ab-4a34-8e82-4406d183abf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:42:32.533424 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2825dcdb-f3ab-4a34-8e82-4406d183abf9 admin admin] PATH_INFO: `/v3/projects/016b97603d7948ff8ca42cb48a09b6b6` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:42:32.544960 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2825dcdb-f3ab-4a34-8e82-4406d183abf9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=016b97603d7948ff8ca42cb48a09b6b6)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:42:32.546926 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2825dcdb-f3ab-4a34-8e82-4406d183abf9 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:42:32.652754 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-2825dcdb-f3ab-4a34-8e82-4406d183abf9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '016b97603d7948ff8ca42cb48a09b6b6', 'request_id': 'req-2825dcdb-f3ab-4a34-8e82-4406d183abf9'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:42:32.655019 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:42:32.715693 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-2825dcdb-f3ab-4a34-8e82-4406d183abf9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '016b97603d7948ff8ca42cb48a09b6b6', 'request_id': 'req-2825dcdb-f3ab-4a34-8e82-4406d183abf9'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:42:32.721846 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-2825dcdb-f3ab-4a34-8e82-4406d183abf9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '016b97603d7948ff8ca42cb48a09b6b6', 'request_id': 'req-2825dcdb-f3ab-4a34-8e82-4406d183abf9'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:42:32.734184 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-2825dcdb-f3ab-4a34-8e82-4406d183abf9 admin admin] The token cache is being invalidate because project 016b97603d7948ff8ca42cb48a09b6b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:42:32.734477 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-2825dcdb-f3ab-4a34-8e82-4406d183abf9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:42:32.735151 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-2825dcdb-f3ab-4a34-8e82-4406d183abf9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:42:32.736940 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1163/2311] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:42:32 2022] DELETE /identity/v3/projects/016b97603d7948ff8ca42cb48a09b6b6 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:42:32.743700 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a2dac5-8f60-438a-86a0-012377bebbbf None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:42:32.744078 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a2dac5-8f60-438a-86a0-012377bebbbf None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:42:32.744367 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a2dac5-8f60-438a-86a0-012377bebbbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:42:32.776315 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:42:32.800029 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-f9a2dac5-8f60-438a-86a0-012377bebbbf None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:42:32.801828 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-f9a2dac5-8f60-438a-86a0-012377bebbbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:42:32.860875 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1149/2312] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:42:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:42:32.868226 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221881f1-3538-4ac4-b610-3f6ca133b444 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:42:32.870310 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221881f1-3538-4ac4-b610-3f6ca133b444 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:42:32.870310 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221881f1-3538-4ac4-b610-3f6ca133b444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:42:32.889882 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-221881f1-3538-4ac4-b610-3f6ca133b444 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:42:32.889882 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-221881f1-3538-4ac4-b610-3f6ca133b444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:42:32.897396 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1164/2313] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:42:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 19:42:32.904920 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a2dac5-8f60-438a-86a0-012377bebbbf None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:42:32.916115 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a2dac5-8f60-438a-86a0-012377bebbbf None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:42:32.920741 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb51b0c1-6eeb-4d1d-ad01-d8c4a4ed9f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:42:32.921437 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb51b0c1-6eeb-4d1d-ad01-d8c4a4ed9f7d admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:42:32.921724 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb51b0c1-6eeb-4d1d-ad01-d8c4a4ed9f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:42:32.922007 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb51b0c1-6eeb-4d1d-ad01-d8c4a4ed9f7d admin admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:42:32.922743 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb51b0c1-6eeb-4d1d-ad01-d8c4a4ed9f7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:42:32.923085 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb51b0c1-6eeb-4d1d-ad01-d8c4a4ed9f7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:42:32.924674 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb51b0c1-6eeb-4d1d-ad01-d8c4a4ed9f7d admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:42:32.941383 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1150/2314] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 2 19:42:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:42:32.942775 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221881f1-3538-4ac4-b610-3f6ca133b444 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:42:32.952073 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221881f1-3538-4ac4-b610-3f6ca133b444 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:42:32.957497 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9591f9-6f99-497a-b763-2149260dc634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:42:32.958160 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9591f9-6f99-497a-b763-2149260dc634 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:42:32.958446 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9591f9-6f99-497a-b763-2149260dc634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:42:32.958723 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9591f9-6f99-497a-b763-2149260dc634 admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:42:32.959441 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9591f9-6f99-497a-b763-2149260dc634 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:42:32.961406 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9591f9-6f99-497a-b763-2149260dc634 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:42:33.053597 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1165/2315] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:42:32 2022] POST /identity/v3/projects => generated 383 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:42:33.057797 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb51b0c1-6eeb-4d1d-ad01-d8c4a4ed9f7d admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:42:33.067879 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb51b0c1-6eeb-4d1d-ad01-d8c4a4ed9f7d admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:42:33.124156 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66a4216-60e4-421f-ac68-5184d4dd8bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:42:33.124775 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66a4216-60e4-421f-ac68-5184d4dd8bd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:42:33.125071 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66a4216-60e4-421f-ac68-5184d4dd8bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:42:33.125354 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66a4216-60e4-421f-ac68-5184d4dd8bd1 admin admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:42:33.126078 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66a4216-60e4-421f-ac68-5184d4dd8bd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:42:33.128066 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66a4216-60e4-421f-ac68-5184d4dd8bd1 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:42:33.224045 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1151/2316] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:42:33 2022] POST /identity/v3/users => generated 292 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:42:33.228235 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9591f9-6f99-497a-b763-2149260dc634 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:42:33.238479 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9591f9-6f99-497a-b763-2149260dc634 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:42:33.243103 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ff6c47-09fd-4834-983a-e12fd4f6a49d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:42:33.243765 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff6c47-09fd-4834-983a-e12fd4f6a49d admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:42:33.244053 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff6c47-09fd-4834-983a-e12fd4f6a49d admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:42:33.244334 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff6c47-09fd-4834-983a-e12fd4f6a49d admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:42:33.245072 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ff6c47-09fd-4834-983a-e12fd4f6a49d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:42:33.246665 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ff6c47-09fd-4834-983a-e12fd4f6a49d admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:42:33.257147 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1166/2317] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:42:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:42:33.260883 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66a4216-60e4-421f-ac68-5184d4dd8bd1 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:42:33.271579 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66a4216-60e4-421f-ac68-5184d4dd8bd1 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:42:33.276122 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac1fff7-7c73-4ad2-82ab-14f85cbb9d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:42:33.276766 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac1fff7-7c73-4ad2-82ab-14f85cbb9d61 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:42:33.277051 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac1fff7-7c73-4ad2-82ab-14f85cbb9d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:42:33.277330 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac1fff7-7c73-4ad2-82ab-14f85cbb9d61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:42:33.278049 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac1fff7-7c73-4ad2-82ab-14f85cbb9d61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:42:33.279663 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac1fff7-7c73-4ad2-82ab-14f85cbb9d61 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:42:33.289668 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1152/2318] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:42:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:42:33.293265 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ff6c47-09fd-4834-983a-e12fd4f6a49d admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:42:33.303076 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ff6c47-09fd-4834-983a-e12fd4f6a49d admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:42:33.307681 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b97f6d-4075-46ad-8fdf-a35d275a250c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:42:33.310866 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b97f6d-4075-46ad-8fdf-a35d275a250c admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:42:33.310866 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b97f6d-4075-46ad-8fdf-a35d275a250c admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:42:33.310866 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b97f6d-4075-46ad-8fdf-a35d275a250c admin admin] PATH_INFO: `/v3/projects/aa2047f8b904428c915e23725332d750/users/01d56b72e0344abcba3af6b9ca77abce/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:42:33.324176 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b97f6d-4075-46ad-8fdf-a35d275a250c admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa2047f8b904428c915e23725332d750, user_id=01d56b72e0344abcba3af6b9ca77abce, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:42:33.326862 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b97f6d-4075-46ad-8fdf-a35d275a250c admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:42:33.416294 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1167/2319] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:42:33 2022] PUT /identity/v3/projects/aa2047f8b904428c915e23725332d750/users/01d56b72e0344abcba3af6b9ca77abce/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:42:33.426680 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aaa67b-af42-474e-83f5-74c99bd5a958 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:42:33.427004 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aaa67b-af42-474e-83f5-74c99bd5a958 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:42:33.427288 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aaa67b-af42-474e-83f5-74c99bd5a958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:42:33.455779 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-27aaa67b-af42-474e-83f5-74c99bd5a958 None None] MFA Rules not processed for user `01d56b72e0344abcba3af6b9ca77abce`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:42:33.457025 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-27aaa67b-af42-474e-83f5-74c99bd5a958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:42:33.505438 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:42:33.523074 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1153/2320] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:42:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 19:42:33.534687 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b97f6d-4075-46ad-8fdf-a35d275a250c admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:42:33.536540 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-16b97f6d-4075-46ad-8fdf-a35d275a250c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:42:33.611131 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b97f6d-4075-46ad-8fdf-a35d275a250c admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:42:33.617290 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba96473-ffad-4a14-97ac-6c6791c825bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:42:33.617940 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba96473-ffad-4a14-97ac-6c6791c825bc service nova] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:42:33.618226 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba96473-ffad-4a14-97ac-6c6791c825bc service nova] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:42:33.618508 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba96473-ffad-4a14-97ac-6c6791c825bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:42:33.626256 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba96473-ffad-4a14-97ac-6c6791c825bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:42:33.627865 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba96473-ffad-4a14-97ac-6c6791c825bc service nova] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:42:33.637646 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1168/2321] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:42:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:42:43.589876 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=5 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:42:43.633230 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=71 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:42:48.325498 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aaa67b-af42-474e-83f5-74c99bd5a958 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:42:48.327000 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-27aaa67b-af42-474e-83f5-74c99bd5a958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:42:48.382070 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aaa67b-af42-474e-83f5-74c99bd5a958 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:42:48.385619 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7a2128-c083-4e22-b358-6a68abcc9123 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:42:48.386259 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7a2128-c083-4e22-b358-6a68abcc9123 service cinder] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:42:48.386543 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7a2128-c083-4e22-b358-6a68abcc9123 service cinder] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:42:48.386848 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7a2128-c083-4e22-b358-6a68abcc9123 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:42:48.388592 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-cf7a2128-c083-4e22-b358-6a68abcc9123 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:42:48.456610 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7a2128-c083-4e22-b358-6a68abcc9123 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:42:48.458048 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7a2128-c083-4e22-b358-6a68abcc9123 service cinder] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:42:48.466798 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1154/2322] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:42:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:42:58.462792 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=53 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:43:24.464477 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba96473-ffad-4a14-97ac-6c6791c825bc service nova] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:43:24.466265 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-dba96473-ffad-4a14-97ac-6c6791c825bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:43:24.521666 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba96473-ffad-4a14-97ac-6c6791c825bc service nova] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:43:24.529759 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d65258-17ab-46d2-847c-d1a5471d18f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:43:24.530441 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d65258-17ab-46d2-847c-d1a5471d18f8 service nova] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:43:24.530893 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d65258-17ab-46d2-847c-d1a5471d18f8 service nova] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:43:24.531034 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d65258-17ab-46d2-847c-d1a5471d18f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:43:24.533017 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-00d65258-17ab-46d2-847c-d1a5471d18f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:43:24.546804 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d65258-17ab-46d2-847c-d1a5471d18f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:43:24.550061 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d65258-17ab-46d2-847c-d1a5471d18f8 service nova] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:43:24.562329 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1169/2323] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:43:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:43:24.899378 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7a2128-c083-4e22-b358-6a68abcc9123 service cinder] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:43:24.942658 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7a2128-c083-4e22-b358-6a68abcc9123 service cinder] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:43:24.949634 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07ffb18-003e-4c5a-b18f-eb2da985e726 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:43:24.950849 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ffb18-003e-4c5a-b18f-eb2da985e726 service nova] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:43:24.950849 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ffb18-003e-4c5a-b18f-eb2da985e726 service nova] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:43:24.951237 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ffb18-003e-4c5a-b18f-eb2da985e726 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:43:24.953324 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-b07ffb18-003e-4c5a-b18f-eb2da985e726 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:43:25.053643 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07ffb18-003e-4c5a-b18f-eb2da985e726 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:43:25.057054 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07ffb18-003e-4c5a-b18f-eb2da985e726 service nova] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:43:25.082654 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1155/2324] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:43:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:43:27.004133 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e313ad37-cc45-4284-b41d-a0c5286098c0 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:43:27.006212 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e313ad37-cc45-4284-b41d-a0c5286098c0 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:43:27.006212 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e313ad37-cc45-4284-b41d-a0c5286098c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:43:27.040602 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-e313ad37-cc45-4284-b41d-a0c5286098c0 None None] MFA Rules not processed for user `a0681f3eff324262b4289bf6d8f4c311`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:43:27.041762 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-e313ad37-cc45-4284-b41d-a0c5286098c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:43:27.104246 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1170/2325] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:43:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:43:27.644892 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07ffb18-003e-4c5a-b18f-eb2da985e726 service nova] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:43:27.646300 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-b07ffb18-003e-4c5a-b18f-eb2da985e726 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:43:27.701438 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07ffb18-003e-4c5a-b18f-eb2da985e726 service nova] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:43:27.705043 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e39a170-6dac-4beb-a15f-3e4b9c9c5781 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:43:27.705681 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e39a170-6dac-4beb-a15f-3e4b9c9c5781 service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:43:27.705968 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e39a170-6dac-4beb-a15f-3e4b9c9c5781 service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:43:27.706249 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e39a170-6dac-4beb-a15f-3e4b9c9c5781 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:43:27.707956 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-5e39a170-6dac-4beb-a15f-3e4b9c9c5781 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:43:27.717301 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e39a170-6dac-4beb-a15f-3e4b9c9c5781 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:43:27.718741 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e39a170-6dac-4beb-a15f-3e4b9c9c5781 service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:43:27.727215 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1156/2326] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:43:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:43:28.134973 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e313ad37-cc45-4284-b41d-a0c5286098c0 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:43:28.136575 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-e313ad37-cc45-4284-b41d-a0c5286098c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:43:28.208062 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e313ad37-cc45-4284-b41d-a0c5286098c0 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:43:28.213517 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48d9d02-100c-4d64-b0d6-0f6b0fe1c38f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73ae837e99764bebbe7a6e06ac9049e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:43:28.214169 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d9d02-100c-4d64-b0d6-0f6b0fe1c38f service placement] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:43:28.214454 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d9d02-100c-4d64-b0d6-0f6b0fe1c38f service placement] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:43:28.214734 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d9d02-100c-4d64-b0d6-0f6b0fe1c38f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:43:28.216526 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-a48d9d02-100c-4d64-b0d6-0f6b0fe1c38f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:43:28.228606 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48d9d02-100c-4d64-b0d6-0f6b0fe1c38f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:43:28.230238 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48d9d02-100c-4d64-b0d6-0f6b0fe1c38f service placement] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:43:28.240675 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1171/2327] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:43:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:43:37.368827 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e39a170-6dac-4beb-a15f-3e4b9c9c5781 service neutron] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:43:37.381865 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e39a170-6dac-4beb-a15f-3e4b9c9c5781 service neutron] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:43:37.387548 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4254cde2-d110-4b46-b7c9-bc47d145c200 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:43:37.388197 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4254cde2-d110-4b46-b7c9-bc47d145c200 service nova] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:43:37.388483 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4254cde2-d110-4b46-b7c9-bc47d145c200 service nova] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:43:37.388801 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4254cde2-d110-4b46-b7c9-bc47d145c200 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:43:37.390995 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-4254cde2-d110-4b46-b7c9-bc47d145c200 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:43:37.403239 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4254cde2-d110-4b46-b7c9-bc47d145c200 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:43:37.404820 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4254cde2-d110-4b46-b7c9-bc47d145c200 service nova] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:43:37.416199 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1157/2328] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:43:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:43:38.236516 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=89 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:43:47.411100 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=84 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:44:54.852232 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c337f8-a02b-4067-a5af-0a93accf42eb None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:44:54.852232 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c337f8-a02b-4067-a5af-0a93accf42eb None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:44:54.852232 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c337f8-a02b-4067-a5af-0a93accf42eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:44:54.889775 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-58c337f8-a02b-4067-a5af-0a93accf42eb None None] MFA Rules not processed for user `04f64813f09348eb91cb7bb2869f7612`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:44:54.891054 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-58c337f8-a02b-4067-a5af-0a93accf42eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:44:54.962981 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1172/2329] 10.222.0.33 () {58 vars in 967 bytes} [Thu Jun 2 19:44:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 19:44:55.042883 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57789067-6c5d-40f5-9c30-31f3c2ca9ee2 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:44:55.043358 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57789067-6c5d-40f5-9c30-31f3c2ca9ee2 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:44:55.043477 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57789067-6c5d-40f5-9c30-31f3c2ca9ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:44:55.061619 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-57789067-6c5d-40f5-9c30-31f3c2ca9ee2 None None] MFA Rules not processed for user `04f64813f09348eb91cb7bb2869f7612`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:44:55.062869 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-57789067-6c5d-40f5-9c30-31f3c2ca9ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:44:55.069884 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1158/2330] 10.222.0.33 () {58 vars in 967 bytes} [Thu Jun 2 19:44:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:44:55.116323 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c337f8-a02b-4067-a5af-0a93accf42eb None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:44:55.118234 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-58c337f8-a02b-4067-a5af-0a93accf42eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:44:55.122713 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57789067-6c5d-40f5-9c30-31f3c2ca9ee2 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:44:55.124144 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-57789067-6c5d-40f5-9c30-31f3c2ca9ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:44:55.216127 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c337f8-a02b-4067-a5af-0a93accf42eb None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:44:55.221141 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57789067-6c5d-40f5-9c30-31f3c2ca9ee2 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:44:55.221907 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ac86d9-4cd8-4205-9dc7-235803b67015 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a59e1d67ff94186a3ebcfcb49938a87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:44:55.222546 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ac86d9-4cd8-4205-9dc7-235803b67015 service swift] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:44:55.222870 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ac86d9-4cd8-4205-9dc7-235803b67015 service swift] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:44:55.223153 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ac86d9-4cd8-4205-9dc7-235803b67015 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:44:55.226114 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635c1d0f-64c3-48ff-b6f8-29e7c14cd349 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a59e1d67ff94186a3ebcfcb49938a87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:44:55.226802 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c1d0f-64c3-48ff-b6f8-29e7c14cd349 service swift] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:44:55.227087 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c1d0f-64c3-48ff-b6f8-29e7c14cd349 service swift] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:44:55.227362 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c1d0f-64c3-48ff-b6f8-29e7c14cd349 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:44:55.231376 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ac86d9-4cd8-4205-9dc7-235803b67015 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:44:55.233705 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ac86d9-4cd8-4205-9dc7-235803b67015 service swift] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:44:55.235329 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635c1d0f-64c3-48ff-b6f8-29e7c14cd349 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:44:55.236883 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635c1d0f-64c3-48ff-b6f8-29e7c14cd349 service swift] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:44:55.241989 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1173/2331] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 2 19:44:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 02 19:44:55.246789 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1159/2332] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 2 19:44:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 02 19:44:56.649581 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ac86d9-4cd8-4205-9dc7-235803b67015 service swift] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:44:56.651092 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-f3ac86d9-4cd8-4205-9dc7-235803b67015 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:44:56.685698 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ac86d9-4cd8-4205-9dc7-235803b67015 service swift] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:44:56.685698 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac0c807-d6b8-43a5-ab79-414fae990247 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:44:56.685698 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac0c807-d6b8-43a5-ab79-414fae990247 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:44:56.685698 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac0c807-d6b8-43a5-ab79-414fae990247 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:44:56.686247 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac0c807-d6b8-43a5-ab79-414fae990247 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:44:56.686247 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-6ac0c807-d6b8-43a5-ab79-414fae990247 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:44:56.705935 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:44:56.728823 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635c1d0f-64c3-48ff-b6f8-29e7c14cd349 service swift] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:44:56.729559 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-635c1d0f-64c3-48ff-b6f8-29e7c14cd349 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:44:56.760799 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac0c807-d6b8-43a5-ab79-414fae990247 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:44:56.762256 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac0c807-d6b8-43a5-ab79-414fae990247 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:44:56.770452 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635c1d0f-64c3-48ff-b6f8-29e7c14cd349 service swift] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:44:56.773799 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1174/2333] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:44:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:44:56.775983 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1160e2c7-e610-4c58-a837-36ca659aa2cd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73ae837e99764bebbe7a6e06ac9049e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:44:56.776641 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1160e2c7-e610-4c58-a837-36ca659aa2cd service placement] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:44:56.776928 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1160e2c7-e610-4c58-a837-36ca659aa2cd service placement] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:44:56.777402 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1160e2c7-e610-4c58-a837-36ca659aa2cd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:44:56.779526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-1160e2c7-e610-4c58-a837-36ca659aa2cd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:44:56.792580 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1160e2c7-e610-4c58-a837-36ca659aa2cd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:44:56.794232 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1160e2c7-e610-4c58-a837-36ca659aa2cd service placement] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:44:56.808372 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1160/2334] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:44:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:45:06.769165 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=22 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:45:06.800344 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=62 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:45:17.098944 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60072977-2960-4e32-a045-0fdc17b583e5 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:45:17.099489 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60072977-2960-4e32-a045-0fdc17b583e5 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:45:17.099598 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60072977-2960-4e32-a045-0fdc17b583e5 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:45:17.100946 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1175/2335] 10.222.0.33 () {54 vars in 912 bytes} [Thu Jun 2 19:45:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 02 19:45:17.106007 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e1d09-aeab-4f1d-83c1-935ff6da7c7a None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:45:17.106428 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e1d09-aeab-4f1d-83c1-935ff6da7c7a None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:45:17.106568 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e1d09-aeab-4f1d-83c1-935ff6da7c7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:45:17.145151 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-a88e1d09-aeab-4f1d-83c1-935ff6da7c7a None None] MFA Rules not processed for user `73ae837e99764bebbe7a6e06ac9049e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:45:17.146380 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-a88e1d09-aeab-4f1d-83c1-935ff6da7c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:45:17.155163 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1161/2336] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Jun 2 19:45:17 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:45:17.179168 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60072977-2960-4e32-a045-0fdc17b583e5 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:45:17.192916 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60072977-2960-4e32-a045-0fdc17b583e5 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:45:17.197910 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6e56bf-aed1-4a45-af67-8af02c9b0c00 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73ae837e99764bebbe7a6e06ac9049e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:45:17.198561 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e56bf-aed1-4a45-af67-8af02c9b0c00 service placement] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:45:17.198896 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e56bf-aed1-4a45-af67-8af02c9b0c00 service placement] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:45:17.199178 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e56bf-aed1-4a45-af67-8af02c9b0c00 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:45:17.208401 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6e56bf-aed1-4a45-af67-8af02c9b0c00 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:45:17.209938 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6e56bf-aed1-4a45-af67-8af02c9b0c00 service placement] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:45:17.220378 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1176/2337] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:45:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:45:20.935297 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75712b7-9c1b-4b70-9e2b-0697dc116e68 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:45:20.935813 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75712b7-9c1b-4b70-9e2b-0697dc116e68 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:45:20.935937 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75712b7-9c1b-4b70-9e2b-0697dc116e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:45:20.963859 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-b75712b7-9c1b-4b70-9e2b-0697dc116e68 None None] MFA Rules not processed for user `85a9b91215d34c68827144968041e700`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:45:20.965059 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-b75712b7-9c1b-4b70-9e2b-0697dc116e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:45:21.021767 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1162/2338] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:45:20 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:45:21.324672 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6e56bf-aed1-4a45-af67-8af02c9b0c00 service placement] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:45:21.334394 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6e56bf-aed1-4a45-af67-8af02c9b0c00 service placement] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:45:21.337894 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a95ecb6-c730-4934-b0ba-67e3b38307d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:45:21.338525 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a95ecb6-c730-4934-b0ba-67e3b38307d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:45:21.338836 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a95ecb6-c730-4934-b0ba-67e3b38307d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:45:21.339116 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a95ecb6-c730-4934-b0ba-67e3b38307d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:45:21.340876 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-0a95ecb6-c730-4934-b0ba-67e3b38307d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:45:21.351071 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a95ecb6-c730-4934-b0ba-67e3b38307d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:45:21.352503 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a95ecb6-c730-4934-b0ba-67e3b38307d8 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:45:21.360334 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1177/2339] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:45:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:45:31.014728 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=37 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:45:31.356860 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=12 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:46:04.147821 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75712b7-9c1b-4b70-9e2b-0697dc116e68 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:46:04.177095 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75712b7-9c1b-4b70-9e2b-0697dc116e68 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:46:04.182540 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd7b288-e687-455a-86d1-567e58dfd362 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:46:04.183213 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd7b288-e687-455a-86d1-567e58dfd362 service nova] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:46:04.183503 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd7b288-e687-455a-86d1-567e58dfd362 service nova] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:46:04.183790 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd7b288-e687-455a-86d1-567e58dfd362 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:46:04.185561 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-efd7b288-e687-455a-86d1-567e58dfd362 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:46:04.253508 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd7b288-e687-455a-86d1-567e58dfd362 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:46:04.255118 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd7b288-e687-455a-86d1-567e58dfd362 service nova] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:46:04.264002 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1163/2340] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:46:04 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:46:14.259820 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=40 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:46:16.372460 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a95ecb6-c730-4934-b0ba-67e3b38307d8 service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:46:16.390931 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a95ecb6-c730-4934-b0ba-67e3b38307d8 service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:46:16.394854 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd4910-4ce4-4fb3-9101-4cd8e91dde3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:46:16.395740 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd4910-4ce4-4fb3-9101-4cd8e91dde3f service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:46:16.396032 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd4910-4ce4-4fb3-9101-4cd8e91dde3f service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:46:16.396310 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd4910-4ce4-4fb3-9101-4cd8e91dde3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:46:16.398085 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-87fd4910-4ce4-4fb3-9101-4cd8e91dde3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:46:16.423614 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fd4910-4ce4-4fb3-9101-4cd8e91dde3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:46:16.425662 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fd4910-4ce4-4fb3-9101-4cd8e91dde3f service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:46:16.436534 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1178/2341] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:46:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:46:26.431700 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=15 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:46:54.737278 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd7b288-e687-455a-86d1-567e58dfd362 service nova] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:46:54.755810 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd7b288-e687-455a-86d1-567e58dfd362 service nova] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:46:54.759357 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46622730-49c4-4333-9ddb-85ecd13210e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:46:54.759987 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46622730-49c4-4333-9ddb-85ecd13210e1 service cinder] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:46:54.760788 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46622730-49c4-4333-9ddb-85ecd13210e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:46:54.761064 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46622730-49c4-4333-9ddb-85ecd13210e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:46:54.762857 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-46622730-49c4-4333-9ddb-85ecd13210e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:46:54.773738 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46622730-49c4-4333-9ddb-85ecd13210e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:46:54.775208 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46622730-49c4-4333-9ddb-85ecd13210e1 service cinder] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:46:54.784992 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1164/2342] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:46:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:46:54.846302 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd4910-4ce4-4fb3-9101-4cd8e91dde3f service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:46:54.855700 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd4910-4ce4-4fb3-9101-4cd8e91dde3f service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:46:54.859237 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ef9d13-66ec-40be-8188-c15f8fde59f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:46:54.859877 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef9d13-66ec-40be-8188-c15f8fde59f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:46:54.860163 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef9d13-66ec-40be-8188-c15f8fde59f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:46:54.860442 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef9d13-66ec-40be-8188-c15f8fde59f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:46:54.862200 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-21ef9d13-66ec-40be-8188-c15f8fde59f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:46:54.875583 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ef9d13-66ec-40be-8188-c15f8fde59f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:46:54.877031 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ef9d13-66ec-40be-8188-c15f8fde59f7 service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:46:54.888292 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1179/2343] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:46:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:47:04.780815 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=10 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:47:04.882932 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=6 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:47:09.460380 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46622730-49c4-4333-9ddb-85ecd13210e1 service cinder] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:47:09.469986 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46622730-49c4-4333-9ddb-85ecd13210e1 service cinder] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:47:09.473554 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b7b8e-9e91-4f44-a4db-af6bf6178c09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:47:09.474188 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b7b8e-9e91-4f44-a4db-af6bf6178c09 service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:47:09.474472 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b7b8e-9e91-4f44-a4db-af6bf6178c09 service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:47:09.474758 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b7b8e-9e91-4f44-a4db-af6bf6178c09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:47:09.476604 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-5a7b7b8e-9e91-4f44-a4db-af6bf6178c09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:47:09.499667 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7b7b8e-9e91-4f44-a4db-af6bf6178c09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:47:09.501094 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7b7b8e-9e91-4f44-a4db-af6bf6178c09 service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:47:09.508930 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1165/2344] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:47:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:47:19.505384 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=11 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:48:18.240223 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ef9d13-66ec-40be-8188-c15f8fde59f7 service cinder] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:48:18.251240 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ef9d13-66ec-40be-8188-c15f8fde59f7 service cinder] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:48:18.256659 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630e7df1-070f-49ea-b793-ab04123b1402 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:48:18.257288 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630e7df1-070f-49ea-b793-ab04123b1402 service nova] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:48:18.257571 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630e7df1-070f-49ea-b793-ab04123b1402 service nova] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:48:18.257846 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630e7df1-070f-49ea-b793-ab04123b1402 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:48:18.265421 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630e7df1-070f-49ea-b793-ab04123b1402 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:48:18.267030 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630e7df1-070f-49ea-b793-ab04123b1402 service nova] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:48:18.275881 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1180/2345] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:48:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:48:28.271941 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=6 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:48:55.496185 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b7b8e-9e91-4f44-a4db-af6bf6178c09 service neutron] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:48:55.505071 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b7b8e-9e91-4f44-a4db-af6bf6178c09 service neutron] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:48:55.508601 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ed367b-79d3-4327-b10e-cee047fec560 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:48:55.509239 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed367b-79d3-4327-b10e-cee047fec560 service cinder] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:48:55.509524 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed367b-79d3-4327-b10e-cee047fec560 service cinder] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:48:55.509803 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed367b-79d3-4327-b10e-cee047fec560 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:48:55.511592 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-50ed367b-79d3-4327-b10e-cee047fec560 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:48:55.522252 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ed367b-79d3-4327-b10e-cee047fec560 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:48:55.523696 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ed367b-79d3-4327-b10e-cee047fec560 service cinder] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:48:55.532879 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1166/2346] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:48:55 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:48:55.609902 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630e7df1-070f-49ea-b793-ab04123b1402 service nova] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:48:55.618983 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630e7df1-070f-49ea-b793-ab04123b1402 service nova] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:48:55.622454 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7d2df7-e0e2-45b5-aed6-908db0032b85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:48:55.623124 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d2df7-e0e2-45b5-aed6-908db0032b85 service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:48:55.623409 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d2df7-e0e2-45b5-aed6-908db0032b85 service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:48:55.623692 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d2df7-e0e2-45b5-aed6-908db0032b85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:48:55.631995 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7d2df7-e0e2-45b5-aed6-908db0032b85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:48:55.633412 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7d2df7-e0e2-45b5-aed6-908db0032b85 service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:48:55.660566 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1181/2347] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:48:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:48:59.007730 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd051d-d45a-44ae-953c-5b5daae17746 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:48:59.008204 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd051d-d45a-44ae-953c-5b5daae17746 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:48:59.014505 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd051d-d45a-44ae-953c-5b5daae17746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:48:59.034008 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-85fd051d-d45a-44ae-953c-5b5daae17746 None None] MFA Rules not processed for user `df62e9ab4d044dcaab00c5d76416e3e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:48:59.035290 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-85fd051d-d45a-44ae-953c-5b5daae17746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:48:59.041929 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1167/2348] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:48:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:49:00.725361 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7d2df7-e0e2-45b5-aed6-908db0032b85 service cinder] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:49:00.736857 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7d2df7-e0e2-45b5-aed6-908db0032b85 service cinder] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:49:00.742096 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2d9448-9a00-4e83-ac15-a241eeaaa052 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73ae837e99764bebbe7a6e06ac9049e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:49:00.742891 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d9448-9a00-4e83-ac15-a241eeaaa052 service placement] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:49:00.743321 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d9448-9a00-4e83-ac15-a241eeaaa052 service placement] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:49:00.743468 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d9448-9a00-4e83-ac15-a241eeaaa052 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:49:00.752785 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2d9448-9a00-4e83-ac15-a241eeaaa052 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:49:00.754384 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2d9448-9a00-4e83-ac15-a241eeaaa052 service placement] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:49:00.767029 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1182/2349] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:49:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:49:06.995700 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd051d-d45a-44ae-953c-5b5daae17746 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:49:07.009850 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd051d-d45a-44ae-953c-5b5daae17746 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:49:07.015540 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8d2bab-e12c-42f9-8c48-54a6b6345840 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:49:07.016558 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8d2bab-e12c-42f9-8c48-54a6b6345840 service nova] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:49:07.016849 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8d2bab-e12c-42f9-8c48-54a6b6345840 service nova] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:49:07.017130 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8d2bab-e12c-42f9-8c48-54a6b6345840 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:49:07.019024 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-6c8d2bab-e12c-42f9-8c48-54a6b6345840 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:49:07.033427 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8d2bab-e12c-42f9-8c48-54a6b6345840 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:49:07.035651 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8d2bab-e12c-42f9-8c48-54a6b6345840 service nova] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:49:07.047958 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1168/2350] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:49:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:49:10.761924 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=19 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:49:12.956296 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2d9448-9a00-4e83-ac15-a241eeaaa052 service placement] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:49:12.968913 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2d9448-9a00-4e83-ac15-a241eeaaa052 service placement] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:49:12.975279 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842dfdc8-7e0e-489a-bf00-1d176337133f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:49:12.975968 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842dfdc8-7e0e-489a-bf00-1d176337133f service nova] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:49:12.976254 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842dfdc8-7e0e-489a-bf00-1d176337133f service nova] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:49:12.976536 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842dfdc8-7e0e-489a-bf00-1d176337133f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:49:12.986344 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842dfdc8-7e0e-489a-bf00-1d176337133f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:49:12.988356 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842dfdc8-7e0e-489a-bf00-1d176337133f service nova] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:49:12.999732 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1183/2351] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:49:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:49:17.042411 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=18 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:49:22.995386 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=6 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:50:02.753936 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8d2bab-e12c-42f9-8c48-54a6b6345840 service nova] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:50:02.764915 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8d2bab-e12c-42f9-8c48-54a6b6345840 service nova] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:50:02.770496 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048dbb0b-d777-4907-8785-938591184b71 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73ae837e99764bebbe7a6e06ac9049e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:50:02.771171 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048dbb0b-d777-4907-8785-938591184b71 service placement] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:50:02.771454 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048dbb0b-d777-4907-8785-938591184b71 service placement] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:50:02.771729 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048dbb0b-d777-4907-8785-938591184b71 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:50:02.780607 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048dbb0b-d777-4907-8785-938591184b71 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:50:02.782255 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048dbb0b-d777-4907-8785-938591184b71 service placement] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:50:02.792697 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1169/2352] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:50:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:50:12.798250 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=6 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:50:55.752519 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842dfdc8-7e0e-489a-bf00-1d176337133f service nova] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:50:55.762627 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842dfdc8-7e0e-489a-bf00-1d176337133f service nova] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:50:55.766221 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75675e1-d546-4fcb-ae1e-84cff8fcdcd1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:50:55.766879 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75675e1-d546-4fcb-ae1e-84cff8fcdcd1 service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:50:55.767163 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75675e1-d546-4fcb-ae1e-84cff8fcdcd1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:50:55.767439 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75675e1-d546-4fcb-ae1e-84cff8fcdcd1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:50:55.776630 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75675e1-d546-4fcb-ae1e-84cff8fcdcd1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:50:55.778099 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75675e1-d546-4fcb-ae1e-84cff8fcdcd1 service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:50:55.786991 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1184/2353] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:50:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:51:05.782914 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=6 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:51:18.437743 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048dbb0b-d777-4907-8785-938591184b71 service placement] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:51:18.439220 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-048dbb0b-d777-4907-8785-938591184b71 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:51:18.464453 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048dbb0b-d777-4907-8785-938591184b71 service placement] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:51:18.468241 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9d2408-35d9-4349-9df9-0c3eaa510728 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:51:18.468889 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9d2408-35d9-4349-9df9-0c3eaa510728 service cinder] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:51:18.469211 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9d2408-35d9-4349-9df9-0c3eaa510728 service cinder] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:51:18.469500 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9d2408-35d9-4349-9df9-0c3eaa510728 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:51:18.479978 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9d2408-35d9-4349-9df9-0c3eaa510728 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:51:18.481443 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9d2408-35d9-4349-9df9-0c3eaa510728 service cinder] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:51:18.492169 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1170/2354] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:51:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:51:18.546573 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75675e1-d546-4fcb-ae1e-84cff8fcdcd1 service cinder] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:51:18.560010 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75675e1-d546-4fcb-ae1e-84cff8fcdcd1 service cinder] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:51:18.565568 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d977fe69-bc8c-4f75-8cb1-3befbe00c137 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:51:18.566226 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d977fe69-bc8c-4f75-8cb1-3befbe00c137 service nova] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:51:18.566629 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d977fe69-bc8c-4f75-8cb1-3befbe00c137 service nova] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:51:18.566974 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d977fe69-bc8c-4f75-8cb1-3befbe00c137 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:51:18.577557 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d977fe69-bc8c-4f75-8cb1-3befbe00c137 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:51:18.579194 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d977fe69-bc8c-4f75-8cb1-3befbe00c137 service nova] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:51:18.607596 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1185/2355] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:51:18 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:51:18.621162 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9d2408-35d9-4349-9df9-0c3eaa510728 service cinder] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:51:18.624032 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9d2408-35d9-4349-9df9-0c3eaa510728 service cinder] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:51:18.627604 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55810f61-03ef-4f9e-bdd5-863ea4e2fd2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:51:18.628611 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55810f61-03ef-4f9e-bdd5-863ea4e2fd2c service cinder] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:51:18.628611 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55810f61-03ef-4f9e-bdd5-863ea4e2fd2c service cinder] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:51:18.628866 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55810f61-03ef-4f9e-bdd5-863ea4e2fd2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:51:18.637686 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55810f61-03ef-4f9e-bdd5-863ea4e2fd2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:51:18.640891 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55810f61-03ef-4f9e-bdd5-863ea4e2fd2c service cinder] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:51:18.648614 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1171/2356] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:51:18 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:51:28.604817 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=13 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:51:28.643934 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=17 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:53:13.459174 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d977fe69-bc8c-4f75-8cb1-3befbe00c137 service nova] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:53:13.468310 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d977fe69-bc8c-4f75-8cb1-3befbe00c137 service nova] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:53:13.471848 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275864ee-387c-4448-a4c4-2d949de3ec88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:53:13.472479 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275864ee-387c-4448-a4c4-2d949de3ec88 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:53:13.472760 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275864ee-387c-4448-a4c4-2d949de3ec88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:53:13.473035 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275864ee-387c-4448-a4c4-2d949de3ec88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:53:13.480030 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275864ee-387c-4448-a4c4-2d949de3ec88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:53:13.481460 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275864ee-387c-4448-a4c4-2d949de3ec88 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:53:13.489779 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1186/2357] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:53:13 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 31 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:53:20.168350 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55810f61-03ef-4f9e-bdd5-863ea4e2fd2c service cinder] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:53:20.220850 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55810f61-03ef-4f9e-bdd5-863ea4e2fd2c service cinder] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:53:20.224487 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ca3bcb-b11b-45a7-ae5b-acad8e7989c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:53:20.225131 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ca3bcb-b11b-45a7-ae5b-acad8e7989c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:53:20.225417 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ca3bcb-b11b-45a7-ae5b-acad8e7989c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:53:20.225697 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ca3bcb-b11b-45a7-ae5b-acad8e7989c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:53:20.227715 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-33ca3bcb-b11b-45a7-ae5b-acad8e7989c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:53:20.308057 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ca3bcb-b11b-45a7-ae5b-acad8e7989c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:53:20.309492 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ca3bcb-b11b-45a7-ae5b-acad8e7989c9 service cinder] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:53:20.328243 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1172/2358] 10.222.0.33 () {60 vars in 1396 bytes} [Thu Jun 2 19:53:20 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3250 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:53:20.407829 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275864ee-387c-4448-a4c4-2d949de3ec88 service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:53:20.417465 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275864ee-387c-4448-a4c4-2d949de3ec88 service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:53:20.421625 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1411a662-1e2f-4126-8335-0ad5b3ac43d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:53:20.422255 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1411a662-1e2f-4126-8335-0ad5b3ac43d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:53:20.422537 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1411a662-1e2f-4126-8335-0ad5b3ac43d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:53:20.422848 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1411a662-1e2f-4126-8335-0ad5b3ac43d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:53:20.431832 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1411a662-1e2f-4126-8335-0ad5b3ac43d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:53:20.433311 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1411a662-1e2f-4126-8335-0ad5b3ac43d3 service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:53:20.442515 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1187/2359] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:53:20 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 02 19:53:20.563310 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ca3bcb-b11b-45a7-ae5b-acad8e7989c9 service cinder] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:53:20.571699 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ca3bcb-b11b-45a7-ae5b-acad8e7989c9 service cinder] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:53:20.575348 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa6d9ee-1b14-43ae-b3fc-e926e11ae8d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:53:20.575987 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa6d9ee-1b14-43ae-b3fc-e926e11ae8d0 service cinder] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:53:20.576274 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa6d9ee-1b14-43ae-b3fc-e926e11ae8d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:53:20.576551 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa6d9ee-1b14-43ae-b3fc-e926e11ae8d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:53:20.643689 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa6d9ee-1b14-43ae-b3fc-e926e11ae8d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:53:20.645171 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa6d9ee-1b14-43ae-b3fc-e926e11ae8d0 service cinder] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:53:20.657000 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1173/2360] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:53:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:53:30.438275 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=12 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:53:30.651618 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=95 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:54:55.849201 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1411a662-1e2f-4126-8335-0ad5b3ac43d3 service cinder] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:54:55.850794 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-1411a662-1e2f-4126-8335-0ad5b3ac43d3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:54:55.892340 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1411a662-1e2f-4126-8335-0ad5b3ac43d3 service cinder] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:54:55.896582 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f629753-2331-4275-b134-9320d82e384f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:54:55.897237 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f629753-2331-4275-b134-9320d82e384f service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:54:55.897522 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f629753-2331-4275-b134-9320d82e384f service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:54:55.897863 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f629753-2331-4275-b134-9320d82e384f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:54:55.899783 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-8f629753-2331-4275-b134-9320d82e384f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:54:55.984834 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f629753-2331-4275-b134-9320d82e384f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:54:55.986276 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f629753-2331-4275-b134-9320d82e384f service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:54:55.995641 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1188/2361] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:54:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:54:56.043735 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa6d9ee-1b14-43ae-b3fc-e926e11ae8d0 service cinder] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:54:56.062106 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa6d9ee-1b14-43ae-b3fc-e926e11ae8d0 service cinder] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:54:56.065713 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077c7526-53b0-4e25-8a96-e42e4748463c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:54:56.066347 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077c7526-53b0-4e25-8a96-e42e4748463c service cinder] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:54:56.066634 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077c7526-53b0-4e25-8a96-e42e4748463c service cinder] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:54:56.066946 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077c7526-53b0-4e25-8a96-e42e4748463c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:54:56.075161 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077c7526-53b0-4e25-8a96-e42e4748463c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:54:56.076634 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077c7526-53b0-4e25-8a96-e42e4748463c service cinder] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:54:56.105437 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1174/2362] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:54:56 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:55:05.991281 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=52 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:55:06.102631 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=18 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:55:06.746968 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f629753-2331-4275-b134-9320d82e384f service cinder] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:06.748685 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-8f629753-2331-4275-b134-9320d82e384f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:06.845918 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f629753-2331-4275-b134-9320d82e384f service cinder] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:06.850919 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3f3a6d-123b-4dcc-a8ac-6c4fbf1c3673 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73ae837e99764bebbe7a6e06ac9049e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:06.851562 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f3a6d-123b-4dcc-a8ac-6c4fbf1c3673 service placement] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:06.851846 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f3a6d-123b-4dcc-a8ac-6c4fbf1c3673 service placement] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:06.852122 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f3a6d-123b-4dcc-a8ac-6c4fbf1c3673 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:06.853967 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-be3f3a6d-123b-4dcc-a8ac-6c4fbf1c3673 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:06.865695 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3f3a6d-123b-4dcc-a8ac-6c4fbf1c3673 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:06.867315 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3f3a6d-123b-4dcc-a8ac-6c4fbf1c3673 service placement] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:06.877303 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1189/2363] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:55:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:55:16.872753 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=44 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:55:21.678960 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077c7526-53b0-4e25-8a96-e42e4748463c service cinder] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:21.680010 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-077c7526-53b0-4e25-8a96-e42e4748463c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:21.842469 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077c7526-53b0-4e25-8a96-e42e4748463c service cinder] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:21.851690 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba0252b-f24e-42a6-be1b-cf9147481f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:21.851690 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba0252b-f24e-42a6-be1b-cf9147481f9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:21.851690 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba0252b-f24e-42a6-be1b-cf9147481f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:21.851690 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba0252b-f24e-42a6-be1b-cf9147481f9c admin admin] PATH_INFO: `/v3/users/01d56b72e0344abcba3af6b9ca77abce` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:21.857893 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba0252b-f24e-42a6-be1b-cf9147481f9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=01d56b72e0344abcba3af6b9ca77abce)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:21.860058 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba0252b-f24e-42a6-be1b-cf9147481f9c admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:22.123592 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-3ba0252b-f24e-42a6-be1b-cf9147481f9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01d56b72e0344abcba3af6b9ca77abce'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:22.131673 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-3ba0252b-f24e-42a6-be1b-cf9147481f9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01d56b72e0344abcba3af6b9ca77abce'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:22.132457 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-3ba0252b-f24e-42a6-be1b-cf9147481f9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01d56b72e0344abcba3af6b9ca77abce'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:22.232883 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-3ba0252b-f24e-42a6-be1b-cf9147481f9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01d56b72e0344abcba3af6b9ca77abce'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:22.233890 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-3ba0252b-f24e-42a6-be1b-cf9147481f9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01d56b72e0344abcba3af6b9ca77abce'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:22.276292 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1175/2364] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:55:21 2022] DELETE /identity/v3/users/01d56b72e0344abcba3af6b9ca77abce => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:55:22.286001 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b8948-93ec-4243-a14b-31eb6deaee79 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:22.286001 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b8948-93ec-4243-a14b-31eb6deaee79 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:22.286001 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b8948-93ec-4243-a14b-31eb6deaee79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:22.334966 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-df2b8948-93ec-4243-a14b-31eb6deaee79 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:55:22.344424 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-df2b8948-93ec-4243-a14b-31eb6deaee79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:22.422494 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1190/2365] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:55:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:55:22.436934 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba0252b-f24e-42a6-be1b-cf9147481f9c admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:22.438517 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-3ba0252b-f24e-42a6-be1b-cf9147481f9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:22.509748 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2b8948-93ec-4243-a14b-31eb6deaee79 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:22.511137 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-df2b8948-93ec-4243-a14b-31eb6deaee79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:22.517061 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba0252b-f24e-42a6-be1b-cf9147481f9c admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:22.521508 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659b30c9-16dc-4c97-8ca9-762081fde410 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:22.525372 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b30c9-16dc-4c97-8ca9-762081fde410 service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:22.525372 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b30c9-16dc-4c97-8ca9-762081fde410 service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:22.525372 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b30c9-16dc-4c97-8ca9-762081fde410 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:22.530570 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:55:22.535523 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659b30c9-16dc-4c97-8ca9-762081fde410 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:22.538224 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659b30c9-16dc-4c97-8ca9-762081fde410 service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:22.561317 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1176/2366] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:55:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:55:22.693979 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2b8948-93ec-4243-a14b-31eb6deaee79 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:22.713531 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295d8d69-9856-487f-a1e2-e0fb657d1963 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:22.713531 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295d8d69-9856-487f-a1e2-e0fb657d1963 service nova] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:22.713531 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295d8d69-9856-487f-a1e2-e0fb657d1963 service nova] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:22.713531 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295d8d69-9856-487f-a1e2-e0fb657d1963 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:22.714247 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-295d8d69-9856-487f-a1e2-e0fb657d1963 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:22.734273 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295d8d69-9856-487f-a1e2-e0fb657d1963 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:22.736013 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295d8d69-9856-487f-a1e2-e0fb657d1963 service nova] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:22.750246 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1191/2367] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:55:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 02 19:55:23.265013 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659b30c9-16dc-4c97-8ca9-762081fde410 service neutron] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:23.267244 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-659b30c9-16dc-4c97-8ca9-762081fde410 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:23.286655 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659b30c9-16dc-4c97-8ca9-762081fde410 service neutron] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:23.326541 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ecf62b-a26b-4db9-a7b0-8394cf7a8076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:23.327678 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ecf62b-a26b-4db9-a7b0-8394cf7a8076 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:23.328209 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ecf62b-a26b-4db9-a7b0-8394cf7a8076 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:23.329596 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ecf62b-a26b-4db9-a7b0-8394cf7a8076 admin admin] PATH_INFO: `/v3/projects/aa2047f8b904428c915e23725332d750` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:23.331354 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ecf62b-a26b-4db9-a7b0-8394cf7a8076 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa2047f8b904428c915e23725332d750)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:23.333454 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ecf62b-a26b-4db9-a7b0-8394cf7a8076 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:23.502655 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-a7ecf62b-a26b-4db9-a7b0-8394cf7a8076 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa2047f8b904428c915e23725332d750', 'request_id': 'req-a7ecf62b-a26b-4db9-a7b0-8394cf7a8076'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:23.638352 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-a7ecf62b-a26b-4db9-a7b0-8394cf7a8076 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa2047f8b904428c915e23725332d750', 'request_id': 'req-a7ecf62b-a26b-4db9-a7b0-8394cf7a8076'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:23.648034 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-a7ecf62b-a26b-4db9-a7b0-8394cf7a8076 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa2047f8b904428c915e23725332d750', 'request_id': 'req-a7ecf62b-a26b-4db9-a7b0-8394cf7a8076'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:23.665277 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-a7ecf62b-a26b-4db9-a7b0-8394cf7a8076 admin admin] The token cache is being invalidate because project aa2047f8b904428c915e23725332d750 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:55:23.665716 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-a7ecf62b-a26b-4db9-a7b0-8394cf7a8076 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:23.674846 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-a7ecf62b-a26b-4db9-a7b0-8394cf7a8076 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:23.681080 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1177/2368] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:55:23 2022] DELETE /identity/v3/projects/aa2047f8b904428c915e23725332d750 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:55:23.683250 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37fd7a6-9f93-4b8e-9175-9477430fae46 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:23.689258 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37fd7a6-9f93-4b8e-9175-9477430fae46 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:23.689258 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37fd7a6-9f93-4b8e-9175-9477430fae46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:23.736553 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-c37fd7a6-9f93-4b8e-9175-9477430fae46 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:55:23.737841 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-c37fd7a6-9f93-4b8e-9175-9477430fae46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:23.799578 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:55:23.811185 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1192/2369] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:55:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:55:23.819979 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ecf62b-a26b-4db9-a7b0-8394cf7a8076 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:23.820718 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8b59e2-8019-4ed6-9daf-cd5b00ffea15 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:23.821186 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8b59e2-8019-4ed6-9daf-cd5b00ffea15 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:23.821643 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8b59e2-8019-4ed6-9daf-cd5b00ffea15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:23.822289 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-a7ecf62b-a26b-4db9-a7b0-8394cf7a8076 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:23.847882 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-ec8b59e2-8019-4ed6-9daf-cd5b00ffea15 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:55:23.849149 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-ec8b59e2-8019-4ed6-9daf-cd5b00ffea15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:23.873982 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1193/2370] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:55:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:55:23.880702 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8b59e2-8019-4ed6-9daf-cd5b00ffea15 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:23.890353 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8b59e2-8019-4ed6-9daf-cd5b00ffea15 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:23.898050 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd459ca-1a94-4af1-b277-4c953b198b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:23.899025 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd459ca-1a94-4af1-b277-4c953b198b60 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:23.899426 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd459ca-1a94-4af1-b277-4c953b198b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:23.899720 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd459ca-1a94-4af1-b277-4c953b198b60 admin admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:23.900615 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd459ca-1a94-4af1-b277-4c953b198b60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:55:23.901068 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd459ca-1a94-4af1-b277-4c953b198b60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:23.904000 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd459ca-1a94-4af1-b277-4c953b198b60 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:23.935393 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1194/2371] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 2 19:55:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:55:23.940904 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd459ca-1a94-4af1-b277-4c953b198b60 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:23.965758 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd459ca-1a94-4af1-b277-4c953b198b60 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:23.984036 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbb1c4d-7ddd-4524-be3e-f907b5d518aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:23.984793 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbb1c4d-7ddd-4524-be3e-f907b5d518aa admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:23.985090 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbb1c4d-7ddd-4524-be3e-f907b5d518aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:23.985375 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbb1c4d-7ddd-4524-be3e-f907b5d518aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:23.986086 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbb1c4d-7ddd-4524-be3e-f907b5d518aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:23.989960 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbb1c4d-7ddd-4524-be3e-f907b5d518aa admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:23.991678 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ecf62b-a26b-4db9-a7b0-8394cf7a8076 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:23.999891 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12728c87-cbff-4a79-96b2-f1e06f534f41 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:24.000975 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12728c87-cbff-4a79-96b2-f1e06f534f41 service nova] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:24.001664 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12728c87-cbff-4a79-96b2-f1e06f534f41 service nova] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:24.002198 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12728c87-cbff-4a79-96b2-f1e06f534f41 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:24.004581 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-12728c87-cbff-4a79-96b2-f1e06f534f41 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:24.021345 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12728c87-cbff-4a79-96b2-f1e06f534f41 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:24.024005 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12728c87-cbff-4a79-96b2-f1e06f534f41 service nova] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:24.049090 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1178/2372] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:55:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:55:24.050940 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12728c87-cbff-4a79-96b2-f1e06f534f41 service nova] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:24.054998 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-12728c87-cbff-4a79-96b2-f1e06f534f41 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:24.140844 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1195/2373] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:55:23 2022] POST /identity/v3/projects => generated 365 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:55:24.144820 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbb1c4d-7ddd-4524-be3e-f907b5d518aa admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:24.158618 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12728c87-cbff-4a79-96b2-f1e06f534f41 service nova] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:24.178370 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:55:24.245859 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab83ff26-aed3-488f-a378-a9bb54b4d8d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:24.248213 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbb1c4d-7ddd-4524-be3e-f907b5d518aa admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:24.249538 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab83ff26-aed3-488f-a378-a9bb54b4d8d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:24.251555 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab83ff26-aed3-488f-a378-a9bb54b4d8d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:24.252638 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab83ff26-aed3-488f-a378-a9bb54b4d8d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:24.254979 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e401d9c7-50f2-492e-b4c9-f5050895cc65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:24.255911 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e401d9c7-50f2-492e-b4c9-f5050895cc65 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:24.256269 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e401d9c7-50f2-492e-b4c9-f5050895cc65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:24.256643 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e401d9c7-50f2-492e-b4c9-f5050895cc65 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:24.257392 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e401d9c7-50f2-492e-b4c9-f5050895cc65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:24.258281 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-ab83ff26-aed3-488f-a378-a9bb54b4d8d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:24.259721 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e401d9c7-50f2-492e-b4c9-f5050895cc65 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:24.288221 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab83ff26-aed3-488f-a378-a9bb54b4d8d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:24.290616 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab83ff26-aed3-488f-a378-a9bb54b4d8d5 service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:24.303502 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1179/2374] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:55:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:55:24.400689 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1196/2375] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:55:24 2022] POST /identity/v3/users => generated 283 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 02 19:55:24.400846 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab83ff26-aed3-488f-a378-a9bb54b4d8d5 service neutron] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:24.419071 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab83ff26-aed3-488f-a378-a9bb54b4d8d5 service neutron] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:24.426306 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ba7a9f-effb-4f44-88f9-4f103390369e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:24.433296 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ba7a9f-effb-4f44-88f9-4f103390369e admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:24.433805 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ba7a9f-effb-4f44-88f9-4f103390369e admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:24.434255 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ba7a9f-effb-4f44-88f9-4f103390369e admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:24.435261 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ba7a9f-effb-4f44-88f9-4f103390369e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:24.438031 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ba7a9f-effb-4f44-88f9-4f103390369e admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:24.464074 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e401d9c7-50f2-492e-b4c9-f5050895cc65 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:24.464698 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1180/2376] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:55:24 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:55:24.476881 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e401d9c7-50f2-492e-b4c9-f5050895cc65 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:24.489651 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccac771c-c583-4a3b-924f-a24d02511ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:24.490588 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccac771c-c583-4a3b-924f-a24d02511ca1 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:24.490996 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccac771c-c583-4a3b-924f-a24d02511ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:24.491351 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccac771c-c583-4a3b-924f-a24d02511ca1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:24.492201 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccac771c-c583-4a3b-924f-a24d02511ca1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:24.494575 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccac771c-c583-4a3b-924f-a24d02511ca1 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:24.518000 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1197/2377] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:55:24 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 02 19:55:24.529255 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ba7a9f-effb-4f44-88f9-4f103390369e admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:24.534539 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ba7a9f-effb-4f44-88f9-4f103390369e admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:24.539712 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72378ee2-9de7-4b8b-b111-a33a62ff68d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:24.540849 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72378ee2-9de7-4b8b-b111-a33a62ff68d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:24.541822 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72378ee2-9de7-4b8b-b111-a33a62ff68d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:24.541822 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72378ee2-9de7-4b8b-b111-a33a62ff68d7 admin admin] PATH_INFO: `/v3/projects/b4a8b74caeb345829da11395bce525ce/users/79dfae0f7cbc4790b1e593d793f48eb0/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:24.561812 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72378ee2-9de7-4b8b-b111-a33a62ff68d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4a8b74caeb345829da11395bce525ce, user_id=79dfae0f7cbc4790b1e593d793f48eb0, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:24.578118 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72378ee2-9de7-4b8b-b111-a33a62ff68d7 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:24.671564 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1181/2378] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:55:24 2022] PUT /identity/v3/projects/b4a8b74caeb345829da11395bce525ce/users/79dfae0f7cbc4790b1e593d793f48eb0/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:55:24.672877 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a7f7d-2cee-4a8b-b88a-750106735ffd None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:24.673180 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a7f7d-2cee-4a8b-b88a-750106735ffd None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:24.673468 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a7f7d-2cee-4a8b-b88a-750106735ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:24.706878 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-f37a7f7d-2cee-4a8b-b88a-750106735ffd None None] MFA Rules not processed for user `79dfae0f7cbc4790b1e593d793f48eb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:55:24.708343 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-f37a7f7d-2cee-4a8b-b88a-750106735ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:24.831715 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1198/2379] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:55:24 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:55:24.836004 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72378ee2-9de7-4b8b-b111-a33a62ff68d7 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:24.942921 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72378ee2-9de7-4b8b-b111-a33a62ff68d7 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:24.963286 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0288084-b99c-44da-8814-a431e4323ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:24.964035 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0288084-b99c-44da-8814-a431e4323ed0 admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:24.964394 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0288084-b99c-44da-8814-a431e4323ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:24.964739 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0288084-b99c-44da-8814-a431e4323ed0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:24.965834 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0288084-b99c-44da-8814-a431e4323ed0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:24.968949 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0288084-b99c-44da-8814-a431e4323ed0 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:25.124101 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1182/2380] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:55:24 2022] POST /identity/v3/projects => generated 365 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:55:25.128230 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37a7f7d-2cee-4a8b-b88a-750106735ffd None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:25.140340 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37a7f7d-2cee-4a8b-b88a-750106735ffd None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:25.155921 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:55:25.161596 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0288084-b99c-44da-8814-a431e4323ed0 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:25.163663 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-c0288084-b99c-44da-8814-a431e4323ed0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:25.257331 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaaa11f-49f1-4398-998a-2486cb501bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:25.258449 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaaa11f-49f1-4398-998a-2486cb501bb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:25.258735 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaaa11f-49f1-4398-998a-2486cb501bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:25.259758 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaaa11f-49f1-4398-998a-2486cb501bb4 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:25.260585 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaaa11f-49f1-4398-998a-2486cb501bb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:25.263602 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaaa11f-49f1-4398-998a-2486cb501bb4 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:25.291889 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0288084-b99c-44da-8814-a431e4323ed0 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:25.297116 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dd2c6f-bbc4-4860-aaa5-9448f6422ec7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73ae837e99764bebbe7a6e06ac9049e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:25.297783 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dd2c6f-bbc4-4860-aaa5-9448f6422ec7 service placement] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:25.298068 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dd2c6f-bbc4-4860-aaa5-9448f6422ec7 service placement] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:25.298347 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dd2c6f-bbc4-4860-aaa5-9448f6422ec7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:25.300176 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-44dd2c6f-bbc4-4860-aaa5-9448f6422ec7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:25.312281 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dd2c6f-bbc4-4860-aaa5-9448f6422ec7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:25.314050 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dd2c6f-bbc4-4860-aaa5-9448f6422ec7 service placement] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:25.321378 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:55:25.326227 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1183/2381] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:55:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:55:25.338498 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dd2c6f-bbc4-4860-aaa5-9448f6422ec7 service placement] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:25.343014 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-44dd2c6f-bbc4-4860-aaa5-9448f6422ec7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:25.360213 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dd2c6f-bbc4-4860-aaa5-9448f6422ec7 service placement] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:25.365278 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94361bf-0784-403b-9d85-26c39a9c2f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:25.366261 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94361bf-0784-403b-9d85-26c39a9c2f3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:25.366725 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94361bf-0784-403b-9d85-26c39a9c2f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:25.367212 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94361bf-0784-403b-9d85-26c39a9c2f3d admin admin] PATH_INFO: `/v3/users/e98d8cc46eb84774bc0ddaf8f1b17690` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:25.368640 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94361bf-0784-403b-9d85-26c39a9c2f3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e98d8cc46eb84774bc0ddaf8f1b17690)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:25.370636 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94361bf-0784-403b-9d85-26c39a9c2f3d admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:25.382456 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1199/2382] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:55:25 2022] POST /identity/v3/users => generated 289 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 02 19:55:25.385920 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaaa11f-49f1-4398-998a-2486cb501bb4 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:25.396973 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaaa11f-49f1-4398-998a-2486cb501bb4 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:25.401790 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054e193c-b701-41d8-b426-9a07cc601788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:25.402545 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e193c-b701-41d8-b426-9a07cc601788 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:25.402950 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e193c-b701-41d8-b426-9a07cc601788 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:25.403306 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e193c-b701-41d8-b426-9a07cc601788 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:25.404113 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054e193c-b701-41d8-b426-9a07cc601788 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:25.406154 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054e193c-b701-41d8-b426-9a07cc601788 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:25.417371 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1200/2383] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 02 19:55:25.422016 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054e193c-b701-41d8-b426-9a07cc601788 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:25.433248 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054e193c-b701-41d8-b426-9a07cc601788 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:25.439543 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cd3108-9933-4492-ab02-b8e93694ef0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:25.440539 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd3108-9933-4492-ab02-b8e93694ef0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:25.440823 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd3108-9933-4492-ab02-b8e93694ef0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:25.441104 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd3108-9933-4492-ab02-b8e93694ef0f admin admin] PATH_INFO: `/v3/projects/f4511db55764442d8455a4936dc05aec/users/ecca3aaeb4fb4fbd8be554957307f9a4/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:25.457992 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cd3108-9933-4492-ab02-b8e93694ef0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4511db55764442d8455a4936dc05aec, user_id=ecca3aaeb4fb4fbd8be554957307f9a4, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:25.460719 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cd3108-9933-4492-ab02-b8e93694ef0f admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:25.536332 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1201/2384] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:55:25 2022] PUT /identity/v3/projects/f4511db55764442d8455a4936dc05aec/users/ecca3aaeb4fb4fbd8be554957307f9a4/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 02 19:55:25.541034 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cd3108-9933-4492-ab02-b8e93694ef0f admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:25.560773 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-e94361bf-0784-403b-9d85-26c39a9c2f3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e98d8cc46eb84774bc0ddaf8f1b17690'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:25.570534 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-e94361bf-0784-403b-9d85-26c39a9c2f3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e98d8cc46eb84774bc0ddaf8f1b17690'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:25.571537 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-e94361bf-0784-403b-9d85-26c39a9c2f3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e98d8cc46eb84774bc0ddaf8f1b17690'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:25.612343 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cd3108-9933-4492-ab02-b8e93694ef0f admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:25.616981 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8888dbd-82b8-4b68-8a6a-62438d9e0af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:25.617642 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8888dbd-82b8-4b68-8a6a-62438d9e0af3 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:25.617928 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8888dbd-82b8-4b68-8a6a-62438d9e0af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:25.618208 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8888dbd-82b8-4b68-8a6a-62438d9e0af3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:25.618972 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8888dbd-82b8-4b68-8a6a-62438d9e0af3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:25.620560 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8888dbd-82b8-4b68-8a6a-62438d9e0af3 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:25.630111 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1202/2385] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:55:25.634152 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8888dbd-82b8-4b68-8a6a-62438d9e0af3 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:25.635855 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-b8888dbd-82b8-4b68-8a6a-62438d9e0af3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:25.636265 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-e94361bf-0784-403b-9d85-26c39a9c2f3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e98d8cc46eb84774bc0ddaf8f1b17690'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:25.636903 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-e94361bf-0784-403b-9d85-26c39a9c2f3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e98d8cc46eb84774bc0ddaf8f1b17690'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:25.648610 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8888dbd-82b8-4b68-8a6a-62438d9e0af3 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:25.653407 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f545d0f3-e37b-4724-a782-9b84e4eb85ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:25.655085 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f545d0f3-e37b-4724-a782-9b84e4eb85ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:25.655381 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f545d0f3-e37b-4724-a782-9b84e4eb85ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:25.655658 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f545d0f3-e37b-4724-a782-9b84e4eb85ac admin admin] PATH_INFO: `/v3/projects/f4511db55764442d8455a4936dc05aec/users/ecca3aaeb4fb4fbd8be554957307f9a4/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:25.657030 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1184/2386] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:55:25 2022] DELETE /identity/v3/users/e98d8cc46eb84774bc0ddaf8f1b17690 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:55:25.657659 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f545d0f3-e37b-4724-a782-9b84e4eb85ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4511db55764442d8455a4936dc05aec, user_id=ecca3aaeb4fb4fbd8be554957307f9a4, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:25.660459 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f545d0f3-e37b-4724-a782-9b84e4eb85ac admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:25.671995 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1203/2387] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:55:25 2022] PUT /identity/v3/projects/f4511db55764442d8455a4936dc05aec/users/ecca3aaeb4fb4fbd8be554957307f9a4/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 02 19:55:25.683470 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9c1cbc-010d-4f43-8d37-23e417b8b934 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:25.683794 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9c1cbc-010d-4f43-8d37-23e417b8b934 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:25.684080 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9c1cbc-010d-4f43-8d37-23e417b8b934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:25.716238 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-4e9c1cbc-010d-4f43-8d37-23e417b8b934 None None] MFA Rules not processed for user `ecca3aaeb4fb4fbd8be554957307f9a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:55:25.717564 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-4e9c1cbc-010d-4f43-8d37-23e417b8b934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:25.813309 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1185/2388] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:55:25 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:55:25.826055 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f545d0f3-e37b-4724-a782-9b84e4eb85ac admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:25.829132 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-f545d0f3-e37b-4724-a782-9b84e4eb85ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:25.886989 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f545d0f3-e37b-4724-a782-9b84e4eb85ac admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:25.890564 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c015d5-59cf-47fa-8ecb-eebe22eb6d4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:25.891231 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c015d5-59cf-47fa-8ecb-eebe22eb6d4e service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:25.897327 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c015d5-59cf-47fa-8ecb-eebe22eb6d4e service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:25.897327 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c015d5-59cf-47fa-8ecb-eebe22eb6d4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:25.898013 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:55:25.901490 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c015d5-59cf-47fa-8ecb-eebe22eb6d4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:25.902953 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c015d5-59cf-47fa-8ecb-eebe22eb6d4e service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:25.912168 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1204/2389] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:55:25 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:55:25.961310 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9c1cbc-010d-4f43-8d37-23e417b8b934 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:25.963467 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-4e9c1cbc-010d-4f43-8d37-23e417b8b934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:26.066120 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:55:26.072309 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9c1cbc-010d-4f43-8d37-23e417b8b934 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:26.078259 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e2cba5-483e-46a4-9945-f2bfdf75917b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:26.079678 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e2cba5-483e-46a4-9945-f2bfdf75917b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:26.079678 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e2cba5-483e-46a4-9945-f2bfdf75917b admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:26.079995 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e2cba5-483e-46a4-9945-f2bfdf75917b admin admin] PATH_INFO: `/v3/projects/ccc7e404c01743078a73e9e3f79c0f22` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:26.081316 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e2cba5-483e-46a4-9945-f2bfdf75917b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccc7e404c01743078a73e9e3f79c0f22)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:26.083427 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e2cba5-483e-46a4-9945-f2bfdf75917b admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:26.171561 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c015d5-59cf-47fa-8ecb-eebe22eb6d4e service cinder] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:26.180036 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c015d5-59cf-47fa-8ecb-eebe22eb6d4e service cinder] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:26.183693 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0800cf34-7429-45ee-a9f1-abc48fc51a77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:26.184329 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0800cf34-7429-45ee-a9f1-abc48fc51a77 service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:26.184668 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0800cf34-7429-45ee-a9f1-abc48fc51a77 service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:26.184951 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0800cf34-7429-45ee-a9f1-abc48fc51a77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:26.186701 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-0800cf34-7429-45ee-a9f1-abc48fc51a77 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:26.224027 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-77e2cba5-483e-46a4-9945-f2bfdf75917b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccc7e404c01743078a73e9e3f79c0f22', 'request_id': 'req-77e2cba5-483e-46a4-9945-f2bfdf75917b'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:26.271063 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0800cf34-7429-45ee-a9f1-abc48fc51a77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:26.273411 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0800cf34-7429-45ee-a9f1-abc48fc51a77 service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:26.285940 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1205/2390] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:55:26 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:55:26.300726 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-77e2cba5-483e-46a4-9945-f2bfdf75917b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccc7e404c01743078a73e9e3f79c0f22', 'request_id': 'req-77e2cba5-483e-46a4-9945-f2bfdf75917b'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:26.308730 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-77e2cba5-483e-46a4-9945-f2bfdf75917b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccc7e404c01743078a73e9e3f79c0f22', 'request_id': 'req-77e2cba5-483e-46a4-9945-f2bfdf75917b'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:26.324170 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-77e2cba5-483e-46a4-9945-f2bfdf75917b admin admin] The token cache is being invalidate because project ccc7e404c01743078a73e9e3f79c0f22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:55:26.324468 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-77e2cba5-483e-46a4-9945-f2bfdf75917b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:26.325147 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-77e2cba5-483e-46a4-9945-f2bfdf75917b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:26.327386 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1186/2391] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:55:25 2022] DELETE /identity/v3/projects/ccc7e404c01743078a73e9e3f79c0f22 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:55:29.128326 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78856687-8f3f-44a4-8529-347becd0b394 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:29.128853 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78856687-8f3f-44a4-8529-347becd0b394 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:29.128969 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78856687-8f3f-44a4-8529-347becd0b394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:29.176794 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-78856687-8f3f-44a4-8529-347becd0b394 None None] MFA Rules not processed for user `a0681f3eff324262b4289bf6d8f4c311`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:55:29.178069 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-78856687-8f3f-44a4-8529-347becd0b394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:29.241711 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1206/2392] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:55:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:55:29.537699 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e2cba5-483e-46a4-9945-f2bfdf75917b admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:29.539258 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-77e2cba5-483e-46a4-9945-f2bfdf75917b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:29.600596 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e2cba5-483e-46a4-9945-f2bfdf75917b admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:29.604232 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549bd0c0-e191-4f85-b9a5-594aba23832e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:29.604877 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549bd0c0-e191-4f85-b9a5-594aba23832e service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:29.605161 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549bd0c0-e191-4f85-b9a5-594aba23832e service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:29.608786 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549bd0c0-e191-4f85-b9a5-594aba23832e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:29.612682 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-549bd0c0-e191-4f85-b9a5-594aba23832e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:29.624125 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549bd0c0-e191-4f85-b9a5-594aba23832e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:29.625972 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549bd0c0-e191-4f85-b9a5-594aba23832e service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:29.635037 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1187/2393] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:55:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:55:29.660360 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78856687-8f3f-44a4-8529-347becd0b394 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:29.661950 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-78856687-8f3f-44a4-8529-347becd0b394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:29.674485 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78856687-8f3f-44a4-8529-347becd0b394 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:29.678383 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b33e88f-896a-4d6d-9298-5492b74f26f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:29.679064 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b33e88f-896a-4d6d-9298-5492b74f26f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:29.679348 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b33e88f-896a-4d6d-9298-5492b74f26f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:29.679624 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b33e88f-896a-4d6d-9298-5492b74f26f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:29.681452 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-4b33e88f-896a-4d6d-9298-5492b74f26f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:29.750768 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:55:29.753151 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b33e88f-896a-4d6d-9298-5492b74f26f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:29.754702 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b33e88f-896a-4d6d-9298-5492b74f26f4 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:29.764846 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1207/2394] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:55:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:55:30.102511 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549bd0c0-e191-4f85-b9a5-594aba23832e service neutron] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:30.104640 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-549bd0c0-e191-4f85-b9a5-594aba23832e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:30.191505 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549bd0c0-e191-4f85-b9a5-594aba23832e service neutron] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:30.197051 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6c2a13-5228-4bab-859b-2525e084e82c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73ae837e99764bebbe7a6e06ac9049e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:30.197716 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6c2a13-5228-4bab-859b-2525e084e82c service placement] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:30.198008 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6c2a13-5228-4bab-859b-2525e084e82c service placement] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:30.198293 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6c2a13-5228-4bab-859b-2525e084e82c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:30.201670 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-8f6c2a13-5228-4bab-859b-2525e084e82c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:30.215605 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6c2a13-5228-4bab-859b-2525e084e82c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:30.217747 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6c2a13-5228-4bab-859b-2525e084e82c service placement] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:30.230125 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1188/2395] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:55:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:55:33.364268 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b33e88f-896a-4d6d-9298-5492b74f26f4 service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:33.365909 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-4b33e88f-896a-4d6d-9298-5492b74f26f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:33.380275 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b33e88f-896a-4d6d-9298-5492b74f26f4 service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:33.385035 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0114e53-2ff4-4c82-966d-1f8ff202dd23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:33.385916 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0114e53-2ff4-4c82-966d-1f8ff202dd23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:33.386206 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0114e53-2ff4-4c82-966d-1f8ff202dd23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:33.386484 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0114e53-2ff4-4c82-966d-1f8ff202dd23 admin admin] PATH_INFO: `/v3/users/85a9b91215d34c68827144968041e700` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:33.401328 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0114e53-2ff4-4c82-966d-1f8ff202dd23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85a9b91215d34c68827144968041e700)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:33.403436 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0114e53-2ff4-4c82-966d-1f8ff202dd23 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:33.585196 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c0114e53-2ff4-4c82-966d-1f8ff202dd23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85a9b91215d34c68827144968041e700'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:33.592456 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c0114e53-2ff4-4c82-966d-1f8ff202dd23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85a9b91215d34c68827144968041e700'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:33.659028 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c0114e53-2ff4-4c82-966d-1f8ff202dd23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85a9b91215d34c68827144968041e700'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:33.659681 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c0114e53-2ff4-4c82-966d-1f8ff202dd23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85a9b91215d34c68827144968041e700'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:33.660324 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c0114e53-2ff4-4c82-966d-1f8ff202dd23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85a9b91215d34c68827144968041e700'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:33.678367 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1208/2396] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:55:33 2022] DELETE /identity/v3/users/85a9b91215d34c68827144968041e700 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:55:33.889269 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6c2a13-5228-4bab-859b-2525e084e82c service placement] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:33.900294 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6c2a13-5228-4bab-859b-2525e084e82c service placement] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:33.917430 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:55:33.957382 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf3c77a-0406-44b6-b6c6-0740d2d9d0fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:33.958246 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf3c77a-0406-44b6-b6c6-0740d2d9d0fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:33.958532 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf3c77a-0406-44b6-b6c6-0740d2d9d0fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:33.958835 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf3c77a-0406-44b6-b6c6-0740d2d9d0fe admin admin] PATH_INFO: `/v3/projects/a661f1650b0a48dca89696dc4846d28c` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:33.960238 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf3c77a-0406-44b6-b6c6-0740d2d9d0fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=a661f1650b0a48dca89696dc4846d28c)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:33.962299 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf3c77a-0406-44b6-b6c6-0740d2d9d0fe admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:34.066087 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-baf3c77a-0406-44b6-b6c6-0740d2d9d0fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a661f1650b0a48dca89696dc4846d28c', 'request_id': 'req-baf3c77a-0406-44b6-b6c6-0740d2d9d0fe'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:34.124114 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-baf3c77a-0406-44b6-b6c6-0740d2d9d0fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a661f1650b0a48dca89696dc4846d28c', 'request_id': 'req-baf3c77a-0406-44b6-b6c6-0740d2d9d0fe'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:34.130086 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-baf3c77a-0406-44b6-b6c6-0740d2d9d0fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a661f1650b0a48dca89696dc4846d28c', 'request_id': 'req-baf3c77a-0406-44b6-b6c6-0740d2d9d0fe'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:34.141208 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-baf3c77a-0406-44b6-b6c6-0740d2d9d0fe admin admin] The token cache is being invalidate because project a661f1650b0a48dca89696dc4846d28c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:55:34.141430 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-baf3c77a-0406-44b6-b6c6-0740d2d9d0fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:34.142083 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-baf3c77a-0406-44b6-b6c6-0740d2d9d0fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:34.143899 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1189/2397] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:55:33 2022] DELETE /identity/v3/projects/a661f1650b0a48dca89696dc4846d28c => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:55:34.148701 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee51e6a-c8ec-4642-98f1-e7c9ff2bd93c None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:34.149092 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee51e6a-c8ec-4642-98f1-e7c9ff2bd93c None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:34.149375 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee51e6a-c8ec-4642-98f1-e7c9ff2bd93c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:34.167272 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-5ee51e6a-c8ec-4642-98f1-e7c9ff2bd93c None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:55:34.168999 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-5ee51e6a-c8ec-4642-98f1-e7c9ff2bd93c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:34.227005 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1209/2398] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:55:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:55:34.235594 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e815d191-23fe-4de1-b301-81035b7bf164 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:34.235888 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e815d191-23fe-4de1-b301-81035b7bf164 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:34.236172 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e815d191-23fe-4de1-b301-81035b7bf164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:34.255433 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-e815d191-23fe-4de1-b301-81035b7bf164 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:55:34.256718 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-e815d191-23fe-4de1-b301-81035b7bf164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:34.265253 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1190/2399] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:55:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:55:34.269893 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee51e6a-c8ec-4642-98f1-e7c9ff2bd93c None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:34.280243 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee51e6a-c8ec-4642-98f1-e7c9ff2bd93c None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:34.285245 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d1ca63-0c24-4f2a-b6d1-9265eddcafd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:34.285787 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d1ca63-0c24-4f2a-b6d1-9265eddcafd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:34.286076 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d1ca63-0c24-4f2a-b6d1-9265eddcafd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:34.286365 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d1ca63-0c24-4f2a-b6d1-9265eddcafd0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:34.287175 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d1ca63-0c24-4f2a-b6d1-9265eddcafd0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:55:34.287496 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d1ca63-0c24-4f2a-b6d1-9265eddcafd0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:34.289123 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d1ca63-0c24-4f2a-b6d1-9265eddcafd0 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:34.303679 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1210/2400] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 2 19:55:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:55:34.308338 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e815d191-23fe-4de1-b301-81035b7bf164 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:34.318669 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e815d191-23fe-4de1-b301-81035b7bf164 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:34.622032 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df490da9-a66d-4f94-8c99-36c6b513c78e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:34.622032 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df490da9-a66d-4f94-8c99-36c6b513c78e admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:34.622032 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df490da9-a66d-4f94-8c99-36c6b513c78e admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:34.622032 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df490da9-a66d-4f94-8c99-36c6b513c78e admin admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:34.622553 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df490da9-a66d-4f94-8c99-36c6b513c78e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:34.622553 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df490da9-a66d-4f94-8c99-36c6b513c78e admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:34.622553 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1191/2401] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:55:34 2022] POST /identity/v3/projects => generated 367 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:55:34.622990 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d1ca63-0c24-4f2a-b6d1-9265eddcafd0 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:34.622990 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d1ca63-0c24-4f2a-b6d1-9265eddcafd0 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:34.622990 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:55:34.622990 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6c2d1d-bf58-407b-b488-9f49d2c76cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:34.623556 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c2d1d-bf58-407b-b488-9f49d2c76cc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:34.623556 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c2d1d-bf58-407b-b488-9f49d2c76cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:34.623556 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c2d1d-bf58-407b-b488-9f49d2c76cc8 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:34.623556 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6c2d1d-bf58-407b-b488-9f49d2c76cc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:34.623556 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6c2d1d-bf58-407b-b488-9f49d2c76cc8 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:34.790563 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1211/2402] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:55:34 2022] POST /identity/v3/users => generated 284 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:55:34.794972 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df490da9-a66d-4f94-8c99-36c6b513c78e admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:34.806198 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df490da9-a66d-4f94-8c99-36c6b513c78e admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:34.811564 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64b0c7a-6345-475d-8c67-8c2ffcfabad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:34.812243 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b0c7a-6345-475d-8c67-8c2ffcfabad4 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:34.812532 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b0c7a-6345-475d-8c67-8c2ffcfabad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:34.812817 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b0c7a-6345-475d-8c67-8c2ffcfabad4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:34.813557 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64b0c7a-6345-475d-8c67-8c2ffcfabad4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:34.815258 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64b0c7a-6345-475d-8c67-8c2ffcfabad4 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:34.826977 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1192/2403] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:55:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:55:34.830923 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6c2d1d-bf58-407b-b488-9f49d2c76cc8 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:34.840883 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6c2d1d-bf58-407b-b488-9f49d2c76cc8 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:34.845493 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72db1e12-341d-4336-8ee1-3b2cf591668c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:34.846191 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72db1e12-341d-4336-8ee1-3b2cf591668c admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:34.846476 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72db1e12-341d-4336-8ee1-3b2cf591668c admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:34.846773 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72db1e12-341d-4336-8ee1-3b2cf591668c admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:34.847563 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72db1e12-341d-4336-8ee1-3b2cf591668c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:34.849442 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72db1e12-341d-4336-8ee1-3b2cf591668c admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:34.862239 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1212/2404] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:55:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:55:34.867005 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64b0c7a-6345-475d-8c67-8c2ffcfabad4 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:34.877933 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64b0c7a-6345-475d-8c67-8c2ffcfabad4 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:34.882633 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7eb2fa-efa7-4ecb-8b47-8733f56a1482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:34.883691 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7eb2fa-efa7-4ecb-8b47-8733f56a1482 admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:34.883978 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7eb2fa-efa7-4ecb-8b47-8733f56a1482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:34.884256 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7eb2fa-efa7-4ecb-8b47-8733f56a1482 admin admin] PATH_INFO: `/v3/projects/093d580083f64766aaea9c764f1b5b00/users/1d2172d85d0a4ec494928f04e9c8cbba/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:34.900569 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7eb2fa-efa7-4ecb-8b47-8733f56a1482 admin admin] RBAC: Authorizing `identity:create_grant(project_id=093d580083f64766aaea9c764f1b5b00, user_id=1d2172d85d0a4ec494928f04e9c8cbba, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:34.903912 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7eb2fa-efa7-4ecb-8b47-8733f56a1482 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:34.968785 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1193/2405] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:55:34 2022] PUT /identity/v3/projects/093d580083f64766aaea9c764f1b5b00/users/1d2172d85d0a4ec494928f04e9c8cbba/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:55:35.028560 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3210630-6401-4bf0-bcdd-6df4a7d42270 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:35.028846 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3210630-6401-4bf0-bcdd-6df4a7d42270 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:35.029128 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3210630-6401-4bf0-bcdd-6df4a7d42270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:35.057638 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-b3210630-6401-4bf0-bcdd-6df4a7d42270 None None] MFA Rules not processed for user `1d2172d85d0a4ec494928f04e9c8cbba`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:55:35.060283 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-b3210630-6401-4bf0-bcdd-6df4a7d42270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:35.146278 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1213/2406] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:55:35 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:55:35.149175 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7eb2fa-efa7-4ecb-8b47-8733f56a1482 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:35.204810 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:55:35.220794 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7eb2fa-efa7-4ecb-8b47-8733f56a1482 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:35.227427 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec39c43-e7e6-473a-b090-7cd86889c7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:35.228518 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec39c43-e7e6-473a-b090-7cd86889c7ab admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:35.228518 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec39c43-e7e6-473a-b090-7cd86889c7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:35.228796 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec39c43-e7e6-473a-b090-7cd86889c7ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:35.229775 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec39c43-e7e6-473a-b090-7cd86889c7ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:35.231811 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec39c43-e7e6-473a-b090-7cd86889c7ab admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:35.329384 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1194/2407] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:55:35 2022] POST /identity/v3/projects => generated 367 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:55:35.332224 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3210630-6401-4bf0-bcdd-6df4a7d42270 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:35.344097 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3210630-6401-4bf0-bcdd-6df4a7d42270 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:35.408103 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dcf01a-82f7-4d10-97aa-85367d28292e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:35.408751 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dcf01a-82f7-4d10-97aa-85367d28292e admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:35.409040 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dcf01a-82f7-4d10-97aa-85367d28292e admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:35.409319 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dcf01a-82f7-4d10-97aa-85367d28292e admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:35.410026 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dcf01a-82f7-4d10-97aa-85367d28292e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:35.411989 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dcf01a-82f7-4d10-97aa-85367d28292e admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:35.516799 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1214/2408] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:55:35 2022] POST /identity/v3/users => generated 290 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:55:35.521189 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec39c43-e7e6-473a-b090-7cd86889c7ab admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:35.540838 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec39c43-e7e6-473a-b090-7cd86889c7ab admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:35.547621 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77467e4c-396d-4e3c-9aea-33f4d5211f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:35.548480 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77467e4c-396d-4e3c-9aea-33f4d5211f58 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:35.548836 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77467e4c-396d-4e3c-9aea-33f4d5211f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:35.549341 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77467e4c-396d-4e3c-9aea-33f4d5211f58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:35.550381 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77467e4c-396d-4e3c-9aea-33f4d5211f58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:35.552188 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77467e4c-396d-4e3c-9aea-33f4d5211f58 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:35.574003 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dcf01a-82f7-4d10-97aa-85367d28292e admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:35.574179 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1195/2409] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:55:35 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:55:35.587872 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dcf01a-82f7-4d10-97aa-85367d28292e admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:35.599400 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a7e06d-d209-4a7d-842b-c97ad9f3e736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:35.601239 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a7e06d-d209-4a7d-842b-c97ad9f3e736 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:35.601732 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a7e06d-d209-4a7d-842b-c97ad9f3e736 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:35.602138 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a7e06d-d209-4a7d-842b-c97ad9f3e736 admin admin] PATH_INFO: `/v3/projects/5c81a3d78868443d81ac0e4ad4f64b27/users/25b3767902624bf38499a1be42d98aab/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:35.620293 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a7e06d-d209-4a7d-842b-c97ad9f3e736 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c81a3d78868443d81ac0e4ad4f64b27, user_id=25b3767902624bf38499a1be42d98aab, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:35.623265 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a7e06d-d209-4a7d-842b-c97ad9f3e736 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:35.690287 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1215/2410] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:55:35 2022] PUT /identity/v3/projects/5c81a3d78868443d81ac0e4ad4f64b27/users/25b3767902624bf38499a1be42d98aab/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 02 19:55:35.693498 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77467e4c-396d-4e3c-9aea-33f4d5211f58 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:35.704857 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77467e4c-396d-4e3c-9aea-33f4d5211f58 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:35.766914 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a066578-dd01-4b9a-8007-94212df38fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:35.767592 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a066578-dd01-4b9a-8007-94212df38fdf admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:35.767871 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a066578-dd01-4b9a-8007-94212df38fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:35.768146 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a066578-dd01-4b9a-8007-94212df38fdf admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:35.769138 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a066578-dd01-4b9a-8007-94212df38fdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:35.770867 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a066578-dd01-4b9a-8007-94212df38fdf admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:35.782097 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1196/2411] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:55:35 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:55:35.786021 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a7e06d-d209-4a7d-842b-c97ad9f3e736 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:35.797800 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a7e06d-d209-4a7d-842b-c97ad9f3e736 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:35.802844 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f1fe14-1dd4-483f-be0f-b0404c662557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:35.803853 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f1fe14-1dd4-483f-be0f-b0404c662557 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:35.804215 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f1fe14-1dd4-483f-be0f-b0404c662557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:35.804568 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f1fe14-1dd4-483f-be0f-b0404c662557 admin admin] PATH_INFO: `/v3/projects/5c81a3d78868443d81ac0e4ad4f64b27/users/25b3767902624bf38499a1be42d98aab/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:35.806630 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f1fe14-1dd4-483f-be0f-b0404c662557 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c81a3d78868443d81ac0e4ad4f64b27, user_id=25b3767902624bf38499a1be42d98aab, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:35.809999 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f1fe14-1dd4-483f-be0f-b0404c662557 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:35.820708 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1216/2412] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:55:35 2022] PUT /identity/v3/projects/5c81a3d78868443d81ac0e4ad4f64b27/users/25b3767902624bf38499a1be42d98aab/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:55:35.830729 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35996566-5f79-43aa-9d1d-afbedb5a1f63 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:35.831057 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35996566-5f79-43aa-9d1d-afbedb5a1f63 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:35.831335 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35996566-5f79-43aa-9d1d-afbedb5a1f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:35.861484 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-35996566-5f79-43aa-9d1d-afbedb5a1f63 None None] MFA Rules not processed for user `25b3767902624bf38499a1be42d98aab`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:55:35.862859 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-35996566-5f79-43aa-9d1d-afbedb5a1f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:35.933334 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1197/2413] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:55:35 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 19:55:35.945113 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f1fe14-1dd4-483f-be0f-b0404c662557 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:35.946800 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-21f1fe14-1dd4-483f-be0f-b0404c662557 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:35.953315 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:55:36.003426 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f1fe14-1dd4-483f-be0f-b0404c662557 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:36.007163 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54204318-81e6-4c73-863d-c0c79efd0323 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:36.007808 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54204318-81e6-4c73-863d-c0c79efd0323 service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:36.008094 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54204318-81e6-4c73-863d-c0c79efd0323 service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:36.008379 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54204318-81e6-4c73-863d-c0c79efd0323 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:36.017396 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54204318-81e6-4c73-863d-c0c79efd0323 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:36.019161 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54204318-81e6-4c73-863d-c0c79efd0323 service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:36.034907 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1217/2414] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:55:35 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:55:36.289079 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35996566-5f79-43aa-9d1d-afbedb5a1f63 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:36.298737 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35996566-5f79-43aa-9d1d-afbedb5a1f63 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:36.302277 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f2ddbf-1815-4b92-b019-5573a7b0dd0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:36.302948 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f2ddbf-1815-4b92-b019-5573a7b0dd0b service cinder] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:36.303236 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f2ddbf-1815-4b92-b019-5573a7b0dd0b service cinder] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:36.303563 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f2ddbf-1815-4b92-b019-5573a7b0dd0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:36.345724 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:55:36.362505 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f2ddbf-1815-4b92-b019-5573a7b0dd0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:36.364322 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f2ddbf-1815-4b92-b019-5573a7b0dd0b service cinder] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:36.374593 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1198/2415] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:55:36 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:55:38.402461 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54204318-81e6-4c73-863d-c0c79efd0323 service cinder] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:38.403893 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-54204318-81e6-4c73-863d-c0c79efd0323 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:38.474434 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54204318-81e6-4c73-863d-c0c79efd0323 service cinder] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:38.478956 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4455b057-569f-4c0d-9d13-a804aed01f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:38.479931 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4455b057-569f-4c0d-9d13-a804aed01f0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:38.480208 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4455b057-569f-4c0d-9d13-a804aed01f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:38.480483 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4455b057-569f-4c0d-9d13-a804aed01f0d admin admin] PATH_INFO: `/v3/users/79dfae0f7cbc4790b1e593d793f48eb0` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:38.481714 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4455b057-569f-4c0d-9d13-a804aed01f0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=79dfae0f7cbc4790b1e593d793f48eb0)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:38.483662 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4455b057-569f-4c0d-9d13-a804aed01f0d admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:38.636574 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-4455b057-569f-4c0d-9d13-a804aed01f0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79dfae0f7cbc4790b1e593d793f48eb0'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:38.643565 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-4455b057-569f-4c0d-9d13-a804aed01f0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79dfae0f7cbc4790b1e593d793f48eb0'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:38.738658 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-4455b057-569f-4c0d-9d13-a804aed01f0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79dfae0f7cbc4790b1e593d793f48eb0'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:38.739517 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-4455b057-569f-4c0d-9d13-a804aed01f0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79dfae0f7cbc4790b1e593d793f48eb0'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:38.742220 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-4455b057-569f-4c0d-9d13-a804aed01f0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79dfae0f7cbc4790b1e593d793f48eb0'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:38.769595 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1218/2416] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:55:38 2022] DELETE /identity/v3/users/79dfae0f7cbc4790b1e593d793f48eb0 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:55:38.773823 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8974990-6a12-4c7b-8f24-d028e15df202 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:38.774313 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8974990-6a12-4c7b-8f24-d028e15df202 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:38.774588 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8974990-6a12-4c7b-8f24-d028e15df202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:38.793951 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-f8974990-6a12-4c7b-8f24-d028e15df202 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:55:38.795299 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-f8974990-6a12-4c7b-8f24-d028e15df202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:38.857894 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1199/2417] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:55:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 19:55:38.867639 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4455b057-569f-4c0d-9d13-a804aed01f0d admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:38.869238 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-4455b057-569f-4c0d-9d13-a804aed01f0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:38.902542 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:55:38.928941 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4455b057-569f-4c0d-9d13-a804aed01f0d admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:38.932746 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e2e2c0-efa5-43db-856e-8cb511b97a72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:38.933958 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e2e2c0-efa5-43db-856e-8cb511b97a72 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:38.934245 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e2e2c0-efa5-43db-856e-8cb511b97a72 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:38.934526 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e2e2c0-efa5-43db-856e-8cb511b97a72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:38.943579 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e2e2c0-efa5-43db-856e-8cb511b97a72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:38.945015 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e2e2c0-efa5-43db-856e-8cb511b97a72 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:38.954624 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1219/2418] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:55:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:55:39.362914 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8974990-6a12-4c7b-8f24-d028e15df202 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:39.364495 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-f8974990-6a12-4c7b-8f24-d028e15df202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:39.381090 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8974990-6a12-4c7b-8f24-d028e15df202 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:39.386168 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784da356-5aa0-4654-9b43-9adda12b3da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:39.387898 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784da356-5aa0-4654-9b43-9adda12b3da3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:39.388556 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784da356-5aa0-4654-9b43-9adda12b3da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:39.388848 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784da356-5aa0-4654-9b43-9adda12b3da3 admin admin] PATH_INFO: `/v3/projects/b4a8b74caeb345829da11395bce525ce` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:39.396934 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784da356-5aa0-4654-9b43-9adda12b3da3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4a8b74caeb345829da11395bce525ce)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:39.396934 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784da356-5aa0-4654-9b43-9adda12b3da3 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:39.501279 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-784da356-5aa0-4654-9b43-9adda12b3da3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4a8b74caeb345829da11395bce525ce', 'request_id': 'req-784da356-5aa0-4654-9b43-9adda12b3da3'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:39.570470 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-784da356-5aa0-4654-9b43-9adda12b3da3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4a8b74caeb345829da11395bce525ce', 'request_id': 'req-784da356-5aa0-4654-9b43-9adda12b3da3'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:39.577110 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-784da356-5aa0-4654-9b43-9adda12b3da3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4a8b74caeb345829da11395bce525ce', 'request_id': 'req-784da356-5aa0-4654-9b43-9adda12b3da3'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:39.588122 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-784da356-5aa0-4654-9b43-9adda12b3da3 admin admin] The token cache is being invalidate because project b4a8b74caeb345829da11395bce525ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:55:39.588409 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-784da356-5aa0-4654-9b43-9adda12b3da3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:39.589029 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-784da356-5aa0-4654-9b43-9adda12b3da3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:39.591169 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1200/2419] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:55:39 2022] DELETE /identity/v3/projects/b4a8b74caeb345829da11395bce525ce => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:55:39.595232 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e2e2c0-efa5-43db-856e-8cb511b97a72 service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:39.604833 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e2e2c0-efa5-43db-856e-8cb511b97a72 service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:39.660785 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab952e5-3b7b-4016-9e3d-cac83981ea4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:39.661582 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab952e5-3b7b-4016-9e3d-cac83981ea4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:39.661872 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab952e5-3b7b-4016-9e3d-cac83981ea4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:39.662152 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab952e5-3b7b-4016-9e3d-cac83981ea4f admin admin] PATH_INFO: `/v3/users/ecca3aaeb4fb4fbd8be554957307f9a4` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:39.663398 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab952e5-3b7b-4016-9e3d-cac83981ea4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecca3aaeb4fb4fbd8be554957307f9a4)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:39.665364 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab952e5-3b7b-4016-9e3d-cac83981ea4f admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:39.849774 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-9ab952e5-3b7b-4016-9e3d-cac83981ea4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecca3aaeb4fb4fbd8be554957307f9a4'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:39.856381 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-9ab952e5-3b7b-4016-9e3d-cac83981ea4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecca3aaeb4fb4fbd8be554957307f9a4'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:39.914973 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-9ab952e5-3b7b-4016-9e3d-cac83981ea4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecca3aaeb4fb4fbd8be554957307f9a4'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:39.915563 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-9ab952e5-3b7b-4016-9e3d-cac83981ea4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecca3aaeb4fb4fbd8be554957307f9a4'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:39.916179 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-9ab952e5-3b7b-4016-9e3d-cac83981ea4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecca3aaeb4fb4fbd8be554957307f9a4'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:39.934644 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1220/2420] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:55:39 2022] DELETE /identity/v3/users/ecca3aaeb4fb4fbd8be554957307f9a4 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:55:40.339228 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784da356-5aa0-4654-9b43-9adda12b3da3 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:40.340681 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-784da356-5aa0-4654-9b43-9adda12b3da3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:40.353878 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784da356-5aa0-4654-9b43-9adda12b3da3 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:40.413494 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dff7b3a-a0f6-4e4d-a8d7-ca1e954e5ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:40.414362 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff7b3a-a0f6-4e4d-a8d7-ca1e954e5ae4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:40.414636 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff7b3a-a0f6-4e4d-a8d7-ca1e954e5ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:40.415018 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff7b3a-a0f6-4e4d-a8d7-ca1e954e5ae4 admin admin] PATH_INFO: `/v3/projects/f4511db55764442d8455a4936dc05aec` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:40.416140 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dff7b3a-a0f6-4e4d-a8d7-ca1e954e5ae4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4511db55764442d8455a4936dc05aec)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:40.418603 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dff7b3a-a0f6-4e4d-a8d7-ca1e954e5ae4 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:40.425417 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:55:40.585079 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-2dff7b3a-a0f6-4e4d-a8d7-ca1e954e5ae4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4511db55764442d8455a4936dc05aec', 'request_id': 'req-2dff7b3a-a0f6-4e4d-a8d7-ca1e954e5ae4'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:40.652245 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-2dff7b3a-a0f6-4e4d-a8d7-ca1e954e5ae4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4511db55764442d8455a4936dc05aec', 'request_id': 'req-2dff7b3a-a0f6-4e4d-a8d7-ca1e954e5ae4'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:40.663779 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-2dff7b3a-a0f6-4e4d-a8d7-ca1e954e5ae4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4511db55764442d8455a4936dc05aec', 'request_id': 'req-2dff7b3a-a0f6-4e4d-a8d7-ca1e954e5ae4'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:40.670283 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-2dff7b3a-a0f6-4e4d-a8d7-ca1e954e5ae4 admin admin] The token cache is being invalidate because project f4511db55764442d8455a4936dc05aec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:55:40.670708 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-2dff7b3a-a0f6-4e4d-a8d7-ca1e954e5ae4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:40.671413 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-2dff7b3a-a0f6-4e4d-a8d7-ca1e954e5ae4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:40.672989 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1201/2421] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:55:40 2022] DELETE /identity/v3/projects/f4511db55764442d8455a4936dc05aec => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:55:40.678230 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422b907e-0bec-4ce6-b0da-0845805675fa None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:40.678795 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422b907e-0bec-4ce6-b0da-0845805675fa None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:40.678966 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422b907e-0bec-4ce6-b0da-0845805675fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:40.726143 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-422b907e-0bec-4ce6-b0da-0845805675fa None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:55:40.727393 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-422b907e-0bec-4ce6-b0da-0845805675fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:40.774298 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:55:40.793481 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1221/2422] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:55:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:55:40.801438 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1151f896-b0a0-4ee3-981f-1fcdb555c09c None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:40.801733 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1151f896-b0a0-4ee3-981f-1fcdb555c09c None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:40.802015 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1151f896-b0a0-4ee3-981f-1fcdb555c09c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:40.821172 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-1151f896-b0a0-4ee3-981f-1fcdb555c09c None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:55:40.822384 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-1151f896-b0a0-4ee3-981f-1fcdb555c09c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:40.830914 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1202/2423] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:55:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:55:40.835303 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422b907e-0bec-4ce6-b0da-0845805675fa None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:40.845175 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422b907e-0bec-4ce6-b0da-0845805675fa None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:40.849958 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63f1066-5067-47d8-a197-ece1e5aae95d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:40.850873 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63f1066-5067-47d8-a197-ece1e5aae95d admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:40.851074 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63f1066-5067-47d8-a197-ece1e5aae95d admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:40.851353 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63f1066-5067-47d8-a197-ece1e5aae95d admin admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:40.852091 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63f1066-5067-47d8-a197-ece1e5aae95d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:55:40.852401 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63f1066-5067-47d8-a197-ece1e5aae95d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:40.854035 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63f1066-5067-47d8-a197-ece1e5aae95d admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:40.867383 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1222/2424] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 2 19:55:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:55:40.872384 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1151f896-b0a0-4ee3-981f-1fcdb555c09c None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:40.882979 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1151f896-b0a0-4ee3-981f-1fcdb555c09c None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:40.888373 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeb0e3a-c323-4651-986f-6916b11840de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:40.889117 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeb0e3a-c323-4651-986f-6916b11840de admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:40.889392 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeb0e3a-c323-4651-986f-6916b11840de admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:40.889675 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeb0e3a-c323-4651-986f-6916b11840de admin admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:40.890401 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeb0e3a-c323-4651-986f-6916b11840de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:40.892800 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeb0e3a-c323-4651-986f-6916b11840de admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:40.997468 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1203/2425] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:55:40 2022] POST /identity/v3/projects => generated 369 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:55:41.001646 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63f1066-5067-47d8-a197-ece1e5aae95d admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:41.012024 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63f1066-5067-47d8-a197-ece1e5aae95d admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:41.075110 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d1decf-d4ba-49d3-8d5c-876cfd2f3eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:41.076129 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d1decf-d4ba-49d3-8d5c-876cfd2f3eed admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:41.076129 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d1decf-d4ba-49d3-8d5c-876cfd2f3eed admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:41.076368 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d1decf-d4ba-49d3-8d5c-876cfd2f3eed admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:41.077062 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d1decf-d4ba-49d3-8d5c-876cfd2f3eed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:41.079078 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d1decf-d4ba-49d3-8d5c-876cfd2f3eed admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:41.085286 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeb0e3a-c323-4651-986f-6916b11840de admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:41.086653 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-bbeb0e3a-c323-4651-986f-6916b11840de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:41.170602 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeb0e3a-c323-4651-986f-6916b11840de admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:41.183012 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1223/2426] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:55:41 2022] POST /identity/v3/users => generated 285 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:55:41.187220 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d1decf-d4ba-49d3-8d5c-876cfd2f3eed admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:41.191547 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd6c2ce-cf51-4146-93e1-305ce65d8429 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:41.192266 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6c2ce-cf51-4146-93e1-305ce65d8429 service nova] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:41.192583 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6c2ce-cf51-4146-93e1-305ce65d8429 service nova] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:41.192910 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6c2ce-cf51-4146-93e1-305ce65d8429 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:41.195001 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-efd6c2ce-cf51-4146-93e1-305ce65d8429 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:41.197677 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d1decf-d4ba-49d3-8d5c-876cfd2f3eed admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:41.202969 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bcd5d2-5f66-46da-8cdc-7c9a9282d797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:41.203580 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bcd5d2-5f66-46da-8cdc-7c9a9282d797 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:41.203861 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bcd5d2-5f66-46da-8cdc-7c9a9282d797 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:41.204140 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bcd5d2-5f66-46da-8cdc-7c9a9282d797 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:41.204905 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bcd5d2-5f66-46da-8cdc-7c9a9282d797 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:41.206502 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bcd5d2-5f66-46da-8cdc-7c9a9282d797 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:41.211538 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd6c2ce-cf51-4146-93e1-305ce65d8429 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:41.213514 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd6c2ce-cf51-4146-93e1-305ce65d8429 service nova] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:41.218606 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1224/2427] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:55:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 02 19:55:41.222074 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bcd5d2-5f66-46da-8cdc-7c9a9282d797 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:41.227975 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1204/2428] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:55:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:55:41.232631 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bcd5d2-5f66-46da-8cdc-7c9a9282d797 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:41.237839 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b141e20e-0151-4b6b-b60f-059de877e635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:41.238553 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b141e20e-0151-4b6b-b60f-059de877e635 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:41.238946 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b141e20e-0151-4b6b-b60f-059de877e635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:41.239233 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b141e20e-0151-4b6b-b60f-059de877e635 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:41.239960 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b141e20e-0151-4b6b-b60f-059de877e635 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:41.241563 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b141e20e-0151-4b6b-b60f-059de877e635 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:41.253181 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1225/2429] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:55:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 02 19:55:41.257234 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd6c2ce-cf51-4146-93e1-305ce65d8429 service nova] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:41.272062 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd6c2ce-cf51-4146-93e1-305ce65d8429 service nova] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:41.274664 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838a1989-12c5-4301-8983-40de7f3d6033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:41.285954 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838a1989-12c5-4301-8983-40de7f3d6033 admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:41.285954 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838a1989-12c5-4301-8983-40de7f3d6033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:41.285954 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838a1989-12c5-4301-8983-40de7f3d6033 admin admin] PATH_INFO: `/v3/projects/6d2db247c1344441ae915a7b3fadc361/users/9ed3ff5f8e314299b1b0b7de45fba043/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:41.302631 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838a1989-12c5-4301-8983-40de7f3d6033 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d2db247c1344441ae915a7b3fadc361, user_id=9ed3ff5f8e314299b1b0b7de45fba043, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:41.307247 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838a1989-12c5-4301-8983-40de7f3d6033 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:41.423867 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1205/2430] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:55:41 2022] PUT /identity/v3/projects/6d2db247c1344441ae915a7b3fadc361/users/9ed3ff5f8e314299b1b0b7de45fba043/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:55:41.434547 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc51463-f485-4088-9588-8bda3d905cc8 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:41.434884 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc51463-f485-4088-9588-8bda3d905cc8 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:41.435169 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc51463-f485-4088-9588-8bda3d905cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:41.462517 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-4bc51463-f485-4088-9588-8bda3d905cc8 None None] MFA Rules not processed for user `9ed3ff5f8e314299b1b0b7de45fba043`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:55:41.463829 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-4bc51463-f485-4088-9588-8bda3d905cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:41.534355 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1226/2431] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:55:41 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 19:55:41.538598 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838a1989-12c5-4301-8983-40de7f3d6033 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:41.577915 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:55:41.612085 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838a1989-12c5-4301-8983-40de7f3d6033 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:41.617294 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36131b77-0536-4745-9bf0-04db3f9484c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:41.617999 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36131b77-0536-4745-9bf0-04db3f9484c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:41.618284 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36131b77-0536-4745-9bf0-04db3f9484c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:41.618564 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36131b77-0536-4745-9bf0-04db3f9484c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:41.619311 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36131b77-0536-4745-9bf0-04db3f9484c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:41.621802 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36131b77-0536-4745-9bf0-04db3f9484c4 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:41.710866 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1206/2432] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:55:41 2022] POST /identity/v3/projects => generated 369 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 02 19:55:41.714830 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc51463-f485-4088-9588-8bda3d905cc8 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:41.729163 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc51463-f485-4088-9588-8bda3d905cc8 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:41.764755 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:55:41.792876 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69c9403-aadf-4228-ada1-e56f6d4b13f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:41.793614 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69c9403-aadf-4228-ada1-e56f6d4b13f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:41.793897 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69c9403-aadf-4228-ada1-e56f6d4b13f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:41.794172 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69c9403-aadf-4228-ada1-e56f6d4b13f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:41.794898 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69c9403-aadf-4228-ada1-e56f6d4b13f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:41.796869 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69c9403-aadf-4228-ada1-e56f6d4b13f5 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:41.878427 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1227/2433] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:55:41 2022] POST /identity/v3/users => generated 291 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:55:41.879709 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36131b77-0536-4745-9bf0-04db3f9484c4 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:41.890834 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36131b77-0536-4745-9bf0-04db3f9484c4 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:41.895666 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb51663-8e17-4374-8f23-6062a4149855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:41.896326 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb51663-8e17-4374-8f23-6062a4149855 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:41.896605 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb51663-8e17-4374-8f23-6062a4149855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:41.896897 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb51663-8e17-4374-8f23-6062a4149855 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:41.897618 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb51663-8e17-4374-8f23-6062a4149855 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:41.899391 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb51663-8e17-4374-8f23-6062a4149855 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:41.910596 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1207/2434] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:55:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:55:41.914436 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69c9403-aadf-4228-ada1-e56f6d4b13f5 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:41.924760 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69c9403-aadf-4228-ada1-e56f6d4b13f5 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:41.929502 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a00025b-6627-4d2a-9b25-394f7e3e438c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:41.937240 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a00025b-6627-4d2a-9b25-394f7e3e438c admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:41.937240 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a00025b-6627-4d2a-9b25-394f7e3e438c admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:41.937240 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a00025b-6627-4d2a-9b25-394f7e3e438c admin admin] PATH_INFO: `/v3/projects/a288104f0af446c597b19fefae77e55f/users/e7a9896da872425783e9db3725b87de3/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:41.945301 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a00025b-6627-4d2a-9b25-394f7e3e438c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a288104f0af446c597b19fefae77e55f, user_id=e7a9896da872425783e9db3725b87de3, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:41.948588 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a00025b-6627-4d2a-9b25-394f7e3e438c admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:42.004118 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1228/2435] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:55:41 2022] PUT /identity/v3/projects/a288104f0af446c597b19fefae77e55f/users/e7a9896da872425783e9db3725b87de3/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:55:42.008189 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb51663-8e17-4374-8f23-6062a4149855 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:42.019052 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb51663-8e17-4374-8f23-6062a4149855 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:42.072655 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd3bb8a-cb16-4e7d-9193-7b17ef00a9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:42.073932 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd3bb8a-cb16-4e7d-9193-7b17ef00a9ff admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:42.074212 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd3bb8a-cb16-4e7d-9193-7b17ef00a9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:42.074484 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd3bb8a-cb16-4e7d-9193-7b17ef00a9ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:42.075231 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd3bb8a-cb16-4e7d-9193-7b17ef00a9ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:42.076810 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd3bb8a-cb16-4e7d-9193-7b17ef00a9ff admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:42.086529 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1208/2436] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:55:42 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:55:42.091294 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a00025b-6627-4d2a-9b25-394f7e3e438c admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:42.101328 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a00025b-6627-4d2a-9b25-394f7e3e438c admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:42.106575 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb349e9e-47f0-4aa8-b5d3-df773c7f0bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:42.107499 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb349e9e-47f0-4aa8-b5d3-df773c7f0bd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:42.107788 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb349e9e-47f0-4aa8-b5d3-df773c7f0bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:42.108057 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb349e9e-47f0-4aa8-b5d3-df773c7f0bd8 admin admin] PATH_INFO: `/v3/projects/a288104f0af446c597b19fefae77e55f/users/e7a9896da872425783e9db3725b87de3/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:42.110013 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb349e9e-47f0-4aa8-b5d3-df773c7f0bd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a288104f0af446c597b19fefae77e55f, user_id=e7a9896da872425783e9db3725b87de3, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:42.112697 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb349e9e-47f0-4aa8-b5d3-df773c7f0bd8 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:42.123812 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1229/2437] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:55:42 2022] PUT /identity/v3/projects/a288104f0af446c597b19fefae77e55f/users/e7a9896da872425783e9db3725b87de3/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:55:42.138834 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685c22c1-7a8b-401c-8373-f7cb42464108 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:42.139127 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685c22c1-7a8b-401c-8373-f7cb42464108 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:42.139404 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685c22c1-7a8b-401c-8373-f7cb42464108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:42.166138 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-685c22c1-7a8b-401c-8373-f7cb42464108 None None] MFA Rules not processed for user `e7a9896da872425783e9db3725b87de3`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:55:42.167928 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-685c22c1-7a8b-401c-8373-f7cb42464108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:42.236406 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1209/2438] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:55:42 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:55:42.247117 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb349e9e-47f0-4aa8-b5d3-df773c7f0bd8 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:42.248892 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-eb349e9e-47f0-4aa8-b5d3-df773c7f0bd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:42.306203 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb349e9e-47f0-4aa8-b5d3-df773c7f0bd8 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:42.310344 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d6eb2a-10c7-4eb7-9058-ebff9e9e6b5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:42.311029 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d6eb2a-10c7-4eb7-9058-ebff9e9e6b5a service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:42.311317 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d6eb2a-10c7-4eb7-9058-ebff9e9e6b5a service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:42.311595 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d6eb2a-10c7-4eb7-9058-ebff9e9e6b5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:42.320536 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d6eb2a-10c7-4eb7-9058-ebff9e9e6b5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:42.321966 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d6eb2a-10c7-4eb7-9058-ebff9e9e6b5a service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:42.331824 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1230/2439] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:55:42 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:55:43.069257 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685c22c1-7a8b-401c-8373-f7cb42464108 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:43.070691 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-685c22c1-7a8b-401c-8373-f7cb42464108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:43.097626 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:55:43.135771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685c22c1-7a8b-401c-8373-f7cb42464108 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:43.141317 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e9e3b1-3960-4694-967f-9144b85bc1dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:43.142457 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e9e3b1-3960-4694-967f-9144b85bc1dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:43.142759 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e9e3b1-3960-4694-967f-9144b85bc1dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:43.143051 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e9e3b1-3960-4694-967f-9144b85bc1dc admin admin] PATH_INFO: `/v3/users/1d2172d85d0a4ec494928f04e9c8cbba` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:43.146919 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e9e3b1-3960-4694-967f-9144b85bc1dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d2172d85d0a4ec494928f04e9c8cbba)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:43.146919 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e9e3b1-3960-4694-967f-9144b85bc1dc admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:43.338633 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-b1e9e3b1-3960-4694-967f-9144b85bc1dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d2172d85d0a4ec494928f04e9c8cbba'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:43.345176 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-b1e9e3b1-3960-4694-967f-9144b85bc1dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d2172d85d0a4ec494928f04e9c8cbba'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:43.345848 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-b1e9e3b1-3960-4694-967f-9144b85bc1dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d2172d85d0a4ec494928f04e9c8cbba'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:43.411114 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-b1e9e3b1-3960-4694-967f-9144b85bc1dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d2172d85d0a4ec494928f04e9c8cbba'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:43.411747 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-b1e9e3b1-3960-4694-967f-9144b85bc1dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d2172d85d0a4ec494928f04e9c8cbba'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:43.431315 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1210/2440] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:55:43 2022] DELETE /identity/v3/users/1d2172d85d0a4ec494928f04e9c8cbba => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:55:43.436426 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308041bb-4dbe-4bda-9928-c6e4784b19a7 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:43.436908 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308041bb-4dbe-4bda-9928-c6e4784b19a7 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:43.437016 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308041bb-4dbe-4bda-9928-c6e4784b19a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:43.454959 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-308041bb-4dbe-4bda-9928-c6e4784b19a7 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:55:43.456168 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-308041bb-4dbe-4bda-9928-c6e4784b19a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:43.517777 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1231/2441] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:55:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:55:43.528058 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e9e3b1-3960-4694-967f-9144b85bc1dc admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:43.529587 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-b1e9e3b1-3960-4694-967f-9144b85bc1dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:43.585761 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e9e3b1-3960-4694-967f-9144b85bc1dc admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:43.589896 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c68053-1f90-4e23-80d4-6aba3581fd4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:43.590540 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c68053-1f90-4e23-80d4-6aba3581fd4a service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:43.590867 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c68053-1f90-4e23-80d4-6aba3581fd4a service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:43.591154 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c68053-1f90-4e23-80d4-6aba3581fd4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:43.601166 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c68053-1f90-4e23-80d4-6aba3581fd4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:43.602696 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c68053-1f90-4e23-80d4-6aba3581fd4a service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:43.616438 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1211/2442] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:55:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 02 19:55:44.041429 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308041bb-4dbe-4bda-9928-c6e4784b19a7 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:44.043502 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-308041bb-4dbe-4bda-9928-c6e4784b19a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:44.058509 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308041bb-4dbe-4bda-9928-c6e4784b19a7 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:44.079030 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bd8320-87ef-4358-8bbc-aa005917f3d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:44.079912 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bd8320-87ef-4358-8bbc-aa005917f3d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:44.080196 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bd8320-87ef-4358-8bbc-aa005917f3d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:44.080500 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bd8320-87ef-4358-8bbc-aa005917f3d4 admin admin] PATH_INFO: `/v3/projects/093d580083f64766aaea9c764f1b5b00` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:44.085625 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bd8320-87ef-4358-8bbc-aa005917f3d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=093d580083f64766aaea9c764f1b5b00)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:44.087579 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bd8320-87ef-4358-8bbc-aa005917f3d4 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:44.119059 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:55:44.193186 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-68bd8320-87ef-4358-8bbc-aa005917f3d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '093d580083f64766aaea9c764f1b5b00', 'request_id': 'req-68bd8320-87ef-4358-8bbc-aa005917f3d4'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:44.272596 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-68bd8320-87ef-4358-8bbc-aa005917f3d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '093d580083f64766aaea9c764f1b5b00', 'request_id': 'req-68bd8320-87ef-4358-8bbc-aa005917f3d4'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:44.279330 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-68bd8320-87ef-4358-8bbc-aa005917f3d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '093d580083f64766aaea9c764f1b5b00', 'request_id': 'req-68bd8320-87ef-4358-8bbc-aa005917f3d4'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:44.290545 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-68bd8320-87ef-4358-8bbc-aa005917f3d4 admin admin] The token cache is being invalidate because project 093d580083f64766aaea9c764f1b5b00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:55:44.290895 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-68bd8320-87ef-4358-8bbc-aa005917f3d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:44.291558 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-68bd8320-87ef-4358-8bbc-aa005917f3d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:44.293645 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1232/2443] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:55:44 2022] DELETE /identity/v3/projects/093d580083f64766aaea9c764f1b5b00 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:55:44.297569 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c68053-1f90-4e23-80d4-6aba3581fd4a service neutron] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:44.307741 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c68053-1f90-4e23-80d4-6aba3581fd4a service neutron] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:44.351244 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:55:44.366391 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977ecbea-93c8-41d7-8688-6d1675dccb40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:44.367305 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ecbea-93c8-41d7-8688-6d1675dccb40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:44.367596 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ecbea-93c8-41d7-8688-6d1675dccb40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:44.367877 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ecbea-93c8-41d7-8688-6d1675dccb40 admin admin] PATH_INFO: `/v3/users/25b3767902624bf38499a1be42d98aab` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:44.369134 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977ecbea-93c8-41d7-8688-6d1675dccb40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25b3767902624bf38499a1be42d98aab)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:44.371114 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977ecbea-93c8-41d7-8688-6d1675dccb40 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:44.608751 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-977ecbea-93c8-41d7-8688-6d1675dccb40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25b3767902624bf38499a1be42d98aab'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:44.616758 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-977ecbea-93c8-41d7-8688-6d1675dccb40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25b3767902624bf38499a1be42d98aab'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:44.617480 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-977ecbea-93c8-41d7-8688-6d1675dccb40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25b3767902624bf38499a1be42d98aab'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:44.702664 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-977ecbea-93c8-41d7-8688-6d1675dccb40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25b3767902624bf38499a1be42d98aab'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:44.703621 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-977ecbea-93c8-41d7-8688-6d1675dccb40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25b3767902624bf38499a1be42d98aab'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:44.726926 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1212/2444] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:55:44 2022] DELETE /identity/v3/users/25b3767902624bf38499a1be42d98aab => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:55:45.196766 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bd8320-87ef-4358-8bbc-aa005917f3d4 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:45.198732 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-68bd8320-87ef-4358-8bbc-aa005917f3d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:45.212940 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bd8320-87ef-4358-8bbc-aa005917f3d4 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:45.273655 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff35dc36-b47d-4a8c-80eb-3197fbf077e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:45.274538 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff35dc36-b47d-4a8c-80eb-3197fbf077e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:45.274852 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff35dc36-b47d-4a8c-80eb-3197fbf077e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:45.275132 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff35dc36-b47d-4a8c-80eb-3197fbf077e9 admin admin] PATH_INFO: `/v3/projects/5c81a3d78868443d81ac0e4ad4f64b27` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:45.276559 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff35dc36-b47d-4a8c-80eb-3197fbf077e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c81a3d78868443d81ac0e4ad4f64b27)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:45.278657 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff35dc36-b47d-4a8c-80eb-3197fbf077e9 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:45.321007 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977ecbea-93c8-41d7-8688-6d1675dccb40 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:45.322546 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-977ecbea-93c8-41d7-8688-6d1675dccb40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:45.336047 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977ecbea-93c8-41d7-8688-6d1675dccb40 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:45.340717 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3b90f8-87c0-4165-b4a3-9be193d8befb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:45.341616 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3b90f8-87c0-4165-b4a3-9be193d8befb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:45.341902 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3b90f8-87c0-4165-b4a3-9be193d8befb admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:45.342180 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3b90f8-87c0-4165-b4a3-9be193d8befb admin admin] PATH_INFO: `/v3/users/9ed3ff5f8e314299b1b0b7de45fba043` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:45.343410 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3b90f8-87c0-4165-b4a3-9be193d8befb admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ed3ff5f8e314299b1b0b7de45fba043)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:45.345333 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3b90f8-87c0-4165-b4a3-9be193d8befb admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:45.428572 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-ff35dc36-b47d-4a8c-80eb-3197fbf077e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c81a3d78868443d81ac0e4ad4f64b27', 'request_id': 'req-ff35dc36-b47d-4a8c-80eb-3197fbf077e9'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:45.508137 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-ff35dc36-b47d-4a8c-80eb-3197fbf077e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c81a3d78868443d81ac0e4ad4f64b27', 'request_id': 'req-ff35dc36-b47d-4a8c-80eb-3197fbf077e9'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:45.514492 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-ff35dc36-b47d-4a8c-80eb-3197fbf077e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c81a3d78868443d81ac0e4ad4f64b27', 'request_id': 'req-ff35dc36-b47d-4a8c-80eb-3197fbf077e9'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:45.531424 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-ff35dc36-b47d-4a8c-80eb-3197fbf077e9 admin admin] The token cache is being invalidate because project 5c81a3d78868443d81ac0e4ad4f64b27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:55:45.531718 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-ff35dc36-b47d-4a8c-80eb-3197fbf077e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:45.532388 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-ff35dc36-b47d-4a8c-80eb-3197fbf077e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:45.534022 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1233/2445] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:55:45 2022] DELETE /identity/v3/projects/5c81a3d78868443d81ac0e4ad4f64b27 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:55:45.538920 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfe223b-2941-40e9-a59a-9dc0e06b90ba None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:45.539310 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfe223b-2941-40e9-a59a-9dc0e06b90ba None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:45.539661 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfe223b-2941-40e9-a59a-9dc0e06b90ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:45.551895 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-6b3b90f8-87c0-4165-b4a3-9be193d8befb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ed3ff5f8e314299b1b0b7de45fba043'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:45.559763 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-6bfe223b-2941-40e9-a59a-9dc0e06b90ba None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:55:45.560637 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-6b3b90f8-87c0-4165-b4a3-9be193d8befb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ed3ff5f8e314299b1b0b7de45fba043'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:45.560979 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-6bfe223b-2941-40e9-a59a-9dc0e06b90ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:45.561469 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-6b3b90f8-87c0-4165-b4a3-9be193d8befb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ed3ff5f8e314299b1b0b7de45fba043'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:45.599988 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:55:45.621678 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1234/2446] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:55:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 19:55:45.629226 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45787cfc-2391-443a-a6cd-23750f297745 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:45.629515 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45787cfc-2391-443a-a6cd-23750f297745 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:45.629795 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45787cfc-2391-443a-a6cd-23750f297745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:45.634005 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-6b3b90f8-87c0-4165-b4a3-9be193d8befb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ed3ff5f8e314299b1b0b7de45fba043'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:45.635029 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-6b3b90f8-87c0-4165-b4a3-9be193d8befb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ed3ff5f8e314299b1b0b7de45fba043'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:45.650623 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-45787cfc-2391-443a-a6cd-23750f297745 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:55:45.651904 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-45787cfc-2391-443a-a6cd-23750f297745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:45.665425 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1235/2447] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:55:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 19:55:45.670641 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45787cfc-2391-443a-a6cd-23750f297745 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:45.670807 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1213/2448] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:55:45 2022] DELETE /identity/v3/users/9ed3ff5f8e314299b1b0b7de45fba043 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 02 19:55:45.676406 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e8e43e-71c1-4e08-89f5-bdcebaa8e807 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:45.676771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e8e43e-71c1-4e08-89f5-bdcebaa8e807 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:45.677398 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e8e43e-71c1-4e08-89f5-bdcebaa8e807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:45.686189 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45787cfc-2391-443a-a6cd-23750f297745 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:45.700271 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-30e8e43e-71c1-4e08-89f5-bdcebaa8e807 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:55:45.701686 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-30e8e43e-71c1-4e08-89f5-bdcebaa8e807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:45.739232 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:55:45.763211 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ab504e-280a-48cb-b4cb-7144a19b137c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:45.764324 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ab504e-280a-48cb-b4cb-7144a19b137c admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:45.764620 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ab504e-280a-48cb-b4cb-7144a19b137c admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:45.764902 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ab504e-280a-48cb-b4cb-7144a19b137c admin admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:45.766978 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ab504e-280a-48cb-b4cb-7144a19b137c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:55:45.767214 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ab504e-280a-48cb-b4cb-7144a19b137c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:45.768855 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ab504e-280a-48cb-b4cb-7144a19b137c admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:45.782818 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1236/2449] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 2 19:55:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:55:45.787542 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ab504e-280a-48cb-b4cb-7144a19b137c admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:45.792595 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1214/2450] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:55:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:55:45.799799 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ab504e-280a-48cb-b4cb-7144a19b137c admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:45.800837 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e8e43e-71c1-4e08-89f5-bdcebaa8e807 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:45.802369 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-30e8e43e-71c1-4e08-89f5-bdcebaa8e807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:45.805287 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585d80e7-c67f-4ee2-86f0-efcd45ec6195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:45.806172 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d80e7-c67f-4ee2-86f0-efcd45ec6195 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:45.806460 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d80e7-c67f-4ee2-86f0-efcd45ec6195 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:45.806793 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d80e7-c67f-4ee2-86f0-efcd45ec6195 admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:45.808000 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585d80e7-c67f-4ee2-86f0-efcd45ec6195 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:45.809906 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585d80e7-c67f-4ee2-86f0-efcd45ec6195 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:45.872748 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e8e43e-71c1-4e08-89f5-bdcebaa8e807 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:45.876434 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d5975f-285c-4cd3-9da5-2bae44370478 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:45.877343 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d5975f-285c-4cd3-9da5-2bae44370478 service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:45.877575 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d5975f-285c-4cd3-9da5-2bae44370478 service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:45.877848 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d5975f-285c-4cd3-9da5-2bae44370478 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:45.889796 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d5975f-285c-4cd3-9da5-2bae44370478 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:45.891291 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d5975f-285c-4cd3-9da5-2bae44370478 service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:45.902328 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1215/2451] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:55:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 02 19:55:45.916480 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1237/2452] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:55:45 2022] POST /identity/v3/projects => generated 383 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:55:45.920739 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d5975f-285c-4cd3-9da5-2bae44370478 service neutron] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:45.934737 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d5975f-285c-4cd3-9da5-2bae44370478 service neutron] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:46.019041 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2de5c64-960a-4c09-95f8-1f428cbc9d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:46.019041 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2de5c64-960a-4c09-95f8-1f428cbc9d70 admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:46.019041 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2de5c64-960a-4c09-95f8-1f428cbc9d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:46.019041 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2de5c64-960a-4c09-95f8-1f428cbc9d70 admin admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:46.019384 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2de5c64-960a-4c09-95f8-1f428cbc9d70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:46.021227 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2de5c64-960a-4c09-95f8-1f428cbc9d70 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:46.137793 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1216/2453] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:55:45 2022] POST /identity/v3/users => generated 292 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:55:46.142029 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585d80e7-c67f-4ee2-86f0-efcd45ec6195 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:46.153125 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585d80e7-c67f-4ee2-86f0-efcd45ec6195 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:46.158228 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ec8def-66ac-4efd-98e9-d00967c7ed8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:46.158946 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ec8def-66ac-4efd-98e9-d00967c7ed8a admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:46.159269 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ec8def-66ac-4efd-98e9-d00967c7ed8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:46.159553 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ec8def-66ac-4efd-98e9-d00967c7ed8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:46.160367 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ec8def-66ac-4efd-98e9-d00967c7ed8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:46.162056 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ec8def-66ac-4efd-98e9-d00967c7ed8a admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:46.190811 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1238/2454] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:55:46 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:55:46.195359 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2de5c64-960a-4c09-95f8-1f428cbc9d70 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:46.210231 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2de5c64-960a-4c09-95f8-1f428cbc9d70 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:46.216638 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d899d2f-ccac-4e05-a705-4d4050212129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:46.217606 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d899d2f-ccac-4e05-a705-4d4050212129 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:46.218187 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d899d2f-ccac-4e05-a705-4d4050212129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:46.218711 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d899d2f-ccac-4e05-a705-4d4050212129 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:46.219747 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d899d2f-ccac-4e05-a705-4d4050212129 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:46.221768 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d899d2f-ccac-4e05-a705-4d4050212129 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:46.234515 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1217/2455] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:55:46 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:55:46.238854 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ec8def-66ac-4efd-98e9-d00967c7ed8a admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:46.249917 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ec8def-66ac-4efd-98e9-d00967c7ed8a admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:46.255371 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5610df4-86be-4937-af49-c7dba190391a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:46.256548 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5610df4-86be-4937-af49-c7dba190391a admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:46.256665 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5610df4-86be-4937-af49-c7dba190391a admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:46.257035 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5610df4-86be-4937-af49-c7dba190391a admin admin] PATH_INFO: `/v3/projects/d1cfed5b12c0402ba50f1a7ad230e891/users/2a838cff9e1c4eedb2cb7fe71b2fbfbe/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:46.277827 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5610df4-86be-4937-af49-c7dba190391a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1cfed5b12c0402ba50f1a7ad230e891, user_id=2a838cff9e1c4eedb2cb7fe71b2fbfbe, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:46.281132 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5610df4-86be-4937-af49-c7dba190391a admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:46.344274 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1239/2456] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:55:46 2022] PUT /identity/v3/projects/d1cfed5b12c0402ba50f1a7ad230e891/users/2a838cff9e1c4eedb2cb7fe71b2fbfbe/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 02 19:55:46.354773 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8bc985-b748-4518-8d48-788229f46925 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:46.355298 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8bc985-b748-4518-8d48-788229f46925 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:46.355372 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8bc985-b748-4518-8d48-788229f46925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:46.386317 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-bd8bc985-b748-4518-8d48-788229f46925 None None] MFA Rules not processed for user `2a838cff9e1c4eedb2cb7fe71b2fbfbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:55:46.388046 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-bd8bc985-b748-4518-8d48-788229f46925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:46.396959 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5610df4-86be-4937-af49-c7dba190391a admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:46.398730 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-f5610df4-86be-4937-af49-c7dba190391a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:46.448914 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:55:46.482347 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1218/2457] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:55:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:55:46.483441 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8bc985-b748-4518-8d48-788229f46925 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:46.487737 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5610df4-86be-4937-af49-c7dba190391a admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:46.494457 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8bc985-b748-4518-8d48-788229f46925 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:46.495037 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23537de-c603-4480-b642-72f9aaa7db7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:46.495919 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23537de-c603-4480-b642-72f9aaa7db7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:46.496322 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23537de-c603-4480-b642-72f9aaa7db7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:46.496501 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23537de-c603-4480-b642-72f9aaa7db7b admin admin] PATH_INFO: `/v3/projects/6d2db247c1344441ae915a7b3fadc361` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:46.497814 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23537de-c603-4480-b642-72f9aaa7db7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d2db247c1344441ae915a7b3fadc361)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:46.499866 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23537de-c603-4480-b642-72f9aaa7db7b admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:46.499975 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ff71ad-2e18-4717-837e-24d866578fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:46.500601 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff71ad-2e18-4717-837e-24d866578fff admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:46.500895 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff71ad-2e18-4717-837e-24d866578fff admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:46.501165 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff71ad-2e18-4717-837e-24d866578fff admin admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:46.501864 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ff71ad-2e18-4717-837e-24d866578fff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:46.503842 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ff71ad-2e18-4717-837e-24d866578fff admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:46.518391 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:55:46.607643 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1219/2458] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:55:46 2022] POST /identity/v3/projects => generated 383 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:55:46.611473 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ff71ad-2e18-4717-837e-24d866578fff admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:46.665479 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f23537de-c603-4480-b642-72f9aaa7db7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d2db247c1344441ae915a7b3fadc361', 'request_id': 'req-f23537de-c603-4480-b642-72f9aaa7db7b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:46.679691 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ff71ad-2e18-4717-837e-24d866578fff admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:46.739213 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7b00ac-d754-47da-98a1-ff21c5fcad19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:46.739936 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b00ac-d754-47da-98a1-ff21c5fcad19 admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:46.740221 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b00ac-d754-47da-98a1-ff21c5fcad19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:46.740493 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b00ac-d754-47da-98a1-ff21c5fcad19 admin admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:46.741186 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7b00ac-d754-47da-98a1-ff21c5fcad19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:46.743125 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7b00ac-d754-47da-98a1-ff21c5fcad19 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:46.759045 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f23537de-c603-4480-b642-72f9aaa7db7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d2db247c1344441ae915a7b3fadc361', 'request_id': 'req-f23537de-c603-4480-b642-72f9aaa7db7b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:46.766609 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f23537de-c603-4480-b642-72f9aaa7db7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d2db247c1344441ae915a7b3fadc361', 'request_id': 'req-f23537de-c603-4480-b642-72f9aaa7db7b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:46.778378 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f23537de-c603-4480-b642-72f9aaa7db7b admin admin] The token cache is being invalidate because project 6d2db247c1344441ae915a7b3fadc361 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:55:46.778663 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f23537de-c603-4480-b642-72f9aaa7db7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:46.779431 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f23537de-c603-4480-b642-72f9aaa7db7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:46.781022 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1240/2459] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:55:46 2022] DELETE /identity/v3/projects/6d2db247c1344441ae915a7b3fadc361 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:55:46.784678 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23537de-c603-4480-b642-72f9aaa7db7b admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:46.785994 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-f23537de-c603-4480-b642-72f9aaa7db7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:46.798215 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23537de-c603-4480-b642-72f9aaa7db7b admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:46.803256 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a3da39-0725-4abc-9017-64ec6f36015d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:46.803857 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a3da39-0725-4abc-9017-64ec6f36015d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:46.804138 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a3da39-0725-4abc-9017-64ec6f36015d admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:46.804412 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a3da39-0725-4abc-9017-64ec6f36015d admin admin] PATH_INFO: `/v3/users/e7a9896da872425783e9db3725b87de3` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:46.805600 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a3da39-0725-4abc-9017-64ec6f36015d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7a9896da872425783e9db3725b87de3)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:46.808150 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a3da39-0725-4abc-9017-64ec6f36015d admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:46.851095 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1220/2460] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:55:46 2022] POST /identity/v3/users => generated 298 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:55:46.858977 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7b00ac-d754-47da-98a1-ff21c5fcad19 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:46.860743 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-cf7b00ac-d754-47da-98a1-ff21c5fcad19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:46.876211 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7b00ac-d754-47da-98a1-ff21c5fcad19 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:46.880579 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b719c1fb-def9-41b0-bc52-7c6e489207a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:46.881265 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b719c1fb-def9-41b0-bc52-7c6e489207a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:46.881549 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b719c1fb-def9-41b0-bc52-7c6e489207a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:46.881876 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b719c1fb-def9-41b0-bc52-7c6e489207a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:46.882780 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b719c1fb-def9-41b0-bc52-7c6e489207a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:46.884398 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b719c1fb-def9-41b0-bc52-7c6e489207a5 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:46.902907 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1221/2461] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:55:46 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 02 19:55:46.907815 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b719c1fb-def9-41b0-bc52-7c6e489207a5 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:46.918883 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b719c1fb-def9-41b0-bc52-7c6e489207a5 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:46.923579 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816d89a4-a431-481f-8e29-164188477e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:46.924466 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816d89a4-a431-481f-8e29-164188477e3e admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:46.924749 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816d89a4-a431-481f-8e29-164188477e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:46.925025 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816d89a4-a431-481f-8e29-164188477e3e admin admin] PATH_INFO: `/v3/projects/8aa19f9ab3c94a30969cbec3d6ea805c/users/95396447fd51488d86be26fc447b99ac/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:46.940417 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816d89a4-a431-481f-8e29-164188477e3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8aa19f9ab3c94a30969cbec3d6ea805c, user_id=95396447fd51488d86be26fc447b99ac, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:46.943098 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816d89a4-a431-481f-8e29-164188477e3e admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:47.018292 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-b2a3da39-0725-4abc-9017-64ec6f36015d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7a9896da872425783e9db3725b87de3'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:47.025124 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-b2a3da39-0725-4abc-9017-64ec6f36015d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7a9896da872425783e9db3725b87de3'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:47.033498 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1222/2462] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:55:46 2022] PUT /identity/v3/projects/8aa19f9ab3c94a30969cbec3d6ea805c/users/95396447fd51488d86be26fc447b99ac/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 02 19:55:47.037289 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816d89a4-a431-481f-8e29-164188477e3e admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:47.096939 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816d89a4-a431-481f-8e29-164188477e3e admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:47.102124 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b423b-5e29-4cfb-b5ca-6003840427ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:47.102823 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b423b-5e29-4cfb-b5ca-6003840427ca admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:47.105502 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:55:47.105502 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b423b-5e29-4cfb-b5ca-6003840427ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:47.105502 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b423b-5e29-4cfb-b5ca-6003840427ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:47.106158 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9b423b-5e29-4cfb-b5ca-6003840427ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:47.107791 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9b423b-5e29-4cfb-b5ca-6003840427ca admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:47.117796 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1223/2463] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:55:47 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 02 19:55:47.122497 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b423b-5e29-4cfb-b5ca-6003840427ca admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:47.125536 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-b2a3da39-0725-4abc-9017-64ec6f36015d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7a9896da872425783e9db3725b87de3'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:47.126155 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-b2a3da39-0725-4abc-9017-64ec6f36015d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7a9896da872425783e9db3725b87de3'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:47.126803 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-b2a3da39-0725-4abc-9017-64ec6f36015d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7a9896da872425783e9db3725b87de3'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:47.132117 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b423b-5e29-4cfb-b5ca-6003840427ca admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:47.137539 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b6e01d-e01b-4a09-8ad8-1a01a1dfbac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:47.138416 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b6e01d-e01b-4a09-8ad8-1a01a1dfbac8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:47.138877 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b6e01d-e01b-4a09-8ad8-1a01a1dfbac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:47.139162 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b6e01d-e01b-4a09-8ad8-1a01a1dfbac8 admin admin] PATH_INFO: `/v3/projects/8aa19f9ab3c94a30969cbec3d6ea805c/users/95396447fd51488d86be26fc447b99ac/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:47.141483 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b6e01d-e01b-4a09-8ad8-1a01a1dfbac8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8aa19f9ab3c94a30969cbec3d6ea805c, user_id=95396447fd51488d86be26fc447b99ac, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:47.144259 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b6e01d-e01b-4a09-8ad8-1a01a1dfbac8 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:47.145499 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1241/2464] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:55:46 2022] DELETE /identity/v3/users/e7a9896da872425783e9db3725b87de3 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 02 19:55:47.155452 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1224/2465] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:55:47 2022] PUT /identity/v3/projects/8aa19f9ab3c94a30969cbec3d6ea805c/users/95396447fd51488d86be26fc447b99ac/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 02 19:55:47.173979 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a708fc8f-f0d9-4d59-8a5d-8ba1afb732dc None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:47.174428 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a708fc8f-f0d9-4d59-8a5d-8ba1afb732dc None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:47.174718 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a708fc8f-f0d9-4d59-8a5d-8ba1afb732dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:47.203049 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-a708fc8f-f0d9-4d59-8a5d-8ba1afb732dc None None] MFA Rules not processed for user `95396447fd51488d86be26fc447b99ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:55:47.204258 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-a708fc8f-f0d9-4d59-8a5d-8ba1afb732dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:47.272289 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:55:47.275916 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1242/2466] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:55:47 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:55:47.286396 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b6e01d-e01b-4a09-8ad8-1a01a1dfbac8 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:47.289378 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-d5b6e01d-e01b-4a09-8ad8-1a01a1dfbac8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:47.347963 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b6e01d-e01b-4a09-8ad8-1a01a1dfbac8 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:47.351668 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6751a6e3-3f17-4a75-ba18-360831c27b36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:47.352316 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6751a6e3-3f17-4a75-ba18-360831c27b36 service cinder] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:47.352604 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6751a6e3-3f17-4a75-ba18-360831c27b36 service cinder] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:47.352882 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6751a6e3-3f17-4a75-ba18-360831c27b36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:47.355288 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-6751a6e3-3f17-4a75-ba18-360831c27b36 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:47.418035 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6751a6e3-3f17-4a75-ba18-360831c27b36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:47.419506 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6751a6e3-3f17-4a75-ba18-360831c27b36 service cinder] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:47.429040 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1225/2467] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:55:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:55:47.555519 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a708fc8f-f0d9-4d59-8a5d-8ba1afb732dc None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:47.558333 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-a708fc8f-f0d9-4d59-8a5d-8ba1afb732dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:47.634942 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a708fc8f-f0d9-4d59-8a5d-8ba1afb732dc None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:47.639438 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df64b3c-dc98-40b3-864c-797cfd72d340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:47.640302 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df64b3c-dc98-40b3-864c-797cfd72d340 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:47.640589 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df64b3c-dc98-40b3-864c-797cfd72d340 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:47.640868 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df64b3c-dc98-40b3-864c-797cfd72d340 admin admin] PATH_INFO: `/v3/projects/a288104f0af446c597b19fefae77e55f` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:47.642433 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df64b3c-dc98-40b3-864c-797cfd72d340 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a288104f0af446c597b19fefae77e55f)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:47.644401 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df64b3c-dc98-40b3-864c-797cfd72d340 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:47.782210 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-3df64b3c-dc98-40b3-864c-797cfd72d340 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a288104f0af446c597b19fefae77e55f', 'request_id': 'req-3df64b3c-dc98-40b3-864c-797cfd72d340'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:47.850308 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-3df64b3c-dc98-40b3-864c-797cfd72d340 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a288104f0af446c597b19fefae77e55f', 'request_id': 'req-3df64b3c-dc98-40b3-864c-797cfd72d340'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:47.856234 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-3df64b3c-dc98-40b3-864c-797cfd72d340 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a288104f0af446c597b19fefae77e55f', 'request_id': 'req-3df64b3c-dc98-40b3-864c-797cfd72d340'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:47.867361 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-3df64b3c-dc98-40b3-864c-797cfd72d340 admin admin] The token cache is being invalidate because project a288104f0af446c597b19fefae77e55f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:55:47.867653 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-3df64b3c-dc98-40b3-864c-797cfd72d340 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:47.868278 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-3df64b3c-dc98-40b3-864c-797cfd72d340 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:47.870255 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1243/2468] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:55:47 2022] DELETE /identity/v3/projects/a288104f0af446c597b19fefae77e55f => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:55:47.875325 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31266827-64b4-4ffe-a306-2dc0be017464 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:47.875624 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31266827-64b4-4ffe-a306-2dc0be017464 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:47.875909 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31266827-64b4-4ffe-a306-2dc0be017464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:47.896827 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-31266827-64b4-4ffe-a306-2dc0be017464 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:55:47.898052 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-31266827-64b4-4ffe-a306-2dc0be017464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:47.964166 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1226/2469] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:55:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 19:55:47.972089 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e73ced-105d-48b1-92f4-84253856e4f2 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:47.972375 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e73ced-105d-48b1-92f4-84253856e4f2 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:47.972649 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e73ced-105d-48b1-92f4-84253856e4f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:47.991223 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-e2e73ced-105d-48b1-92f4-84253856e4f2 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:55:47.992435 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-e2e73ced-105d-48b1-92f4-84253856e4f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:48.001380 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1244/2470] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:55:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:55:48.005587 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31266827-64b4-4ffe-a306-2dc0be017464 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:48.012401 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:55:48.016498 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31266827-64b4-4ffe-a306-2dc0be017464 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:48.021147 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a973754d-9a3d-4939-8336-61b72c666c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:48.021805 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a973754d-9a3d-4939-8336-61b72c666c17 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:48.022094 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a973754d-9a3d-4939-8336-61b72c666c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:48.022375 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a973754d-9a3d-4939-8336-61b72c666c17 admin admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:48.023172 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a973754d-9a3d-4939-8336-61b72c666c17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:55:48.023484 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a973754d-9a3d-4939-8336-61b72c666c17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:48.025120 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a973754d-9a3d-4939-8336-61b72c666c17 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:48.037132 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1227/2471] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 2 19:55:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:55:48.041957 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e73ced-105d-48b1-92f4-84253856e4f2 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:48.051219 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e73ced-105d-48b1-92f4-84253856e4f2 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:48.055823 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d02c91-cf4d-41a4-bd65-e26cf339d863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:48.056471 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d02c91-cf4d-41a4-bd65-e26cf339d863 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:48.056754 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d02c91-cf4d-41a4-bd65-e26cf339d863 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:48.057029 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d02c91-cf4d-41a4-bd65-e26cf339d863 admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:48.057714 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d02c91-cf4d-41a4-bd65-e26cf339d863 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:48.059657 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d02c91-cf4d-41a4-bd65-e26cf339d863 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:48.158273 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1245/2472] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:55:48 2022] POST /identity/v3/projects => generated 381 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:55:48.159335 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a973754d-9a3d-4939-8336-61b72c666c17 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:48.172324 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a973754d-9a3d-4939-8336-61b72c666c17 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:48.227894 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124e17c3-d612-4d68-99e5-53a14c39c59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:48.228570 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e17c3-d612-4d68-99e5-53a14c39c59c admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:48.229432 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e17c3-d612-4d68-99e5-53a14c39c59c admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:48.229712 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e17c3-d612-4d68-99e5-53a14c39c59c admin admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:48.230414 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124e17c3-d612-4d68-99e5-53a14c39c59c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:48.232387 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124e17c3-d612-4d68-99e5-53a14c39c59c admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:48.346460 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1228/2473] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:55:48 2022] POST /identity/v3/users => generated 291 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:55:48.350499 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d02c91-cf4d-41a4-bd65-e26cf339d863 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:48.360767 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d02c91-cf4d-41a4-bd65-e26cf339d863 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:48.365225 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad027b1-2a8f-4b8e-becf-d923d57e2969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:48.365879 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad027b1-2a8f-4b8e-becf-d923d57e2969 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:48.366166 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad027b1-2a8f-4b8e-becf-d923d57e2969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:48.366446 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad027b1-2a8f-4b8e-becf-d923d57e2969 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:48.367238 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad027b1-2a8f-4b8e-becf-d923d57e2969 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:48.368931 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad027b1-2a8f-4b8e-becf-d923d57e2969 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:48.379152 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1246/2474] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:55:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:55:48.382857 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124e17c3-d612-4d68-99e5-53a14c39c59c admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:48.393741 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124e17c3-d612-4d68-99e5-53a14c39c59c admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:48.398414 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61066163-e42e-4be7-9a2a-91dded058d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:48.399107 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61066163-e42e-4be7-9a2a-91dded058d18 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:48.399396 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61066163-e42e-4be7-9a2a-91dded058d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:48.399677 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61066163-e42e-4be7-9a2a-91dded058d18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:48.400424 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61066163-e42e-4be7-9a2a-91dded058d18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:48.402645 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61066163-e42e-4be7-9a2a-91dded058d18 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:48.412781 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1229/2475] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:55:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:55:48.416795 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad027b1-2a8f-4b8e-becf-d923d57e2969 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:48.426532 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad027b1-2a8f-4b8e-becf-d923d57e2969 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:48.431087 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3a99ce-5902-46e6-844b-6b760fd2d1c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:48.431963 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3a99ce-5902-46e6-844b-6b760fd2d1c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:48.432246 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3a99ce-5902-46e6-844b-6b760fd2d1c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:48.432521 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3a99ce-5902-46e6-844b-6b760fd2d1c8 admin admin] PATH_INFO: `/v3/projects/f0817ef18e0b491e8518e1a42dcfed97/users/2cde22927b234b20b137629931aaaa8f/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:48.446516 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3a99ce-5902-46e6-844b-6b760fd2d1c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0817ef18e0b491e8518e1a42dcfed97, user_id=2cde22927b234b20b137629931aaaa8f, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:48.449691 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3a99ce-5902-46e6-844b-6b760fd2d1c8 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:48.507194 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1247/2476] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:55:48 2022] PUT /identity/v3/projects/f0817ef18e0b491e8518e1a42dcfed97/users/2cde22927b234b20b137629931aaaa8f/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:55:48.517725 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ada1f-4baa-4d8a-8135-14dd0c40977f None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:48.518049 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ada1f-4baa-4d8a-8135-14dd0c40977f None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:48.518335 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ada1f-4baa-4d8a-8135-14dd0c40977f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:48.545041 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-c85ada1f-4baa-4d8a-8135-14dd0c40977f None None] MFA Rules not processed for user `2cde22927b234b20b137629931aaaa8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:55:48.546303 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-c85ada1f-4baa-4d8a-8135-14dd0c40977f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:48.609652 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1230/2477] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:55:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:55:48.614265 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3a99ce-5902-46e6-844b-6b760fd2d1c8 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:48.644379 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:55:48.676047 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3a99ce-5902-46e6-844b-6b760fd2d1c8 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:48.680679 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e900cca3-79d1-42c3-b421-dbefdb861c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:48.681436 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900cca3-79d1-42c3-b421-dbefdb861c3d admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:48.682355 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900cca3-79d1-42c3-b421-dbefdb861c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:48.682632 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900cca3-79d1-42c3-b421-dbefdb861c3d admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:48.683381 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e900cca3-79d1-42c3-b421-dbefdb861c3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:48.685275 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e900cca3-79d1-42c3-b421-dbefdb861c3d admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:48.787280 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1248/2478] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:55:48 2022] POST /identity/v3/projects => generated 381 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:55:48.791302 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85ada1f-4baa-4d8a-8135-14dd0c40977f None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:48.801756 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85ada1f-4baa-4d8a-8135-14dd0c40977f None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:48.856059 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05448ddd-44ef-460a-8f07-4a51e9b737fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:48.856723 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05448ddd-44ef-460a-8f07-4a51e9b737fe admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:48.857005 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05448ddd-44ef-460a-8f07-4a51e9b737fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:48.857307 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05448ddd-44ef-460a-8f07-4a51e9b737fe admin admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:48.857988 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05448ddd-44ef-460a-8f07-4a51e9b737fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:48.859926 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05448ddd-44ef-460a-8f07-4a51e9b737fe admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:48.866411 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:55:48.967549 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1231/2479] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:55:48 2022] POST /identity/v3/users => generated 297 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:55:48.968744 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e900cca3-79d1-42c3-b421-dbefdb861c3d admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:48.979570 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e900cca3-79d1-42c3-b421-dbefdb861c3d admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:48.984534 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711f03d9-8573-4a1d-b015-2a3a2bfc7e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:48.985309 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f03d9-8573-4a1d-b015-2a3a2bfc7e70 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:48.985651 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f03d9-8573-4a1d-b015-2a3a2bfc7e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:48.985937 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f03d9-8573-4a1d-b015-2a3a2bfc7e70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:48.986669 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711f03d9-8573-4a1d-b015-2a3a2bfc7e70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:48.988316 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711f03d9-8573-4a1d-b015-2a3a2bfc7e70 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:49.003018 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1249/2480] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:55:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:55:49.008258 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05448ddd-44ef-460a-8f07-4a51e9b737fe admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:49.019405 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05448ddd-44ef-460a-8f07-4a51e9b737fe admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:49.024058 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326e0f3c-7149-4815-b324-516e36cf5167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:49.024934 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e0f3c-7149-4815-b324-516e36cf5167 admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:49.025220 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e0f3c-7149-4815-b324-516e36cf5167 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:49.025500 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e0f3c-7149-4815-b324-516e36cf5167 admin admin] PATH_INFO: `/v3/projects/1adfdb2abb71494bbc824f175a501cd3/users/5cffee45a1724600a728967819b23e80/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:49.039833 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326e0f3c-7149-4815-b324-516e36cf5167 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1adfdb2abb71494bbc824f175a501cd3, user_id=5cffee45a1724600a728967819b23e80, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:49.042450 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326e0f3c-7149-4815-b324-516e36cf5167 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:49.119971 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1232/2481] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:55:49 2022] PUT /identity/v3/projects/1adfdb2abb71494bbc824f175a501cd3/users/5cffee45a1724600a728967819b23e80/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:55:49.124137 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711f03d9-8573-4a1d-b015-2a3a2bfc7e70 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:49.135293 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711f03d9-8573-4a1d-b015-2a3a2bfc7e70 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:49.195134 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635083ad-a0f3-4518-965f-b323b7f3205f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:49.195813 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635083ad-a0f3-4518-965f-b323b7f3205f admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:49.196100 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635083ad-a0f3-4518-965f-b323b7f3205f admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:49.196380 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635083ad-a0f3-4518-965f-b323b7f3205f admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:49.197111 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635083ad-a0f3-4518-965f-b323b7f3205f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:49.199257 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635083ad-a0f3-4518-965f-b323b7f3205f admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:49.209471 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1250/2482] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:55:49 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:55:49.213891 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326e0f3c-7149-4815-b324-516e36cf5167 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:49.229026 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326e0f3c-7149-4815-b324-516e36cf5167 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:49.233668 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871bbebe-df26-4f2f-bdb0-e5931fbd743e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:49.234548 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871bbebe-df26-4f2f-bdb0-e5931fbd743e admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:49.234864 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871bbebe-df26-4f2f-bdb0-e5931fbd743e admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:49.235495 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871bbebe-df26-4f2f-bdb0-e5931fbd743e admin admin] PATH_INFO: `/v3/projects/1adfdb2abb71494bbc824f175a501cd3/users/5cffee45a1724600a728967819b23e80/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:49.237441 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871bbebe-df26-4f2f-bdb0-e5931fbd743e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1adfdb2abb71494bbc824f175a501cd3, user_id=5cffee45a1724600a728967819b23e80, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:49.240072 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871bbebe-df26-4f2f-bdb0-e5931fbd743e admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:49.250691 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1233/2483] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:55:49 2022] PUT /identity/v3/projects/1adfdb2abb71494bbc824f175a501cd3/users/5cffee45a1724600a728967819b23e80/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:55:49.262405 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e7640-d9fc-4048-aec9-a02367feb148 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:49.262699 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e7640-d9fc-4048-aec9-a02367feb148 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:49.263026 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e7640-d9fc-4048-aec9-a02367feb148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:49.290203 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-fd9e7640-d9fc-4048-aec9-a02367feb148 None None] MFA Rules not processed for user `5cffee45a1724600a728967819b23e80`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:55:49.291598 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-fd9e7640-d9fc-4048-aec9-a02367feb148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:49.360477 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1251/2484] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:55:49 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:55:49.372503 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871bbebe-df26-4f2f-bdb0-e5931fbd743e admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:49.374175 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-871bbebe-df26-4f2f-bdb0-e5931fbd743e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:49.430811 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871bbebe-df26-4f2f-bdb0-e5931fbd743e admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:49.435240 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6a9c26-ab48-40c0-9530-9428dd094735 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:49.435911 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a9c26-ab48-40c0-9530-9428dd094735 service cinder] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:49.436202 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a9c26-ab48-40c0-9530-9428dd094735 service cinder] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:49.436476 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a9c26-ab48-40c0-9530-9428dd094735 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:49.446049 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6a9c26-ab48-40c0-9530-9428dd094735 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:49.447570 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6a9c26-ab48-40c0-9530-9428dd094735 service cinder] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:49.457193 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1234/2485] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:55:49 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 02 19:55:50.453886 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9e7640-d9fc-4048-aec9-a02367feb148 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:50.463799 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9e7640-d9fc-4048-aec9-a02367feb148 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:50.468440 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d48db2-0a04-4595-b4d5-e24114faea79 tempest-VolumeQuotaClassesTest-1873665499 tempest-VolumeQuotaClassesTest-1873665499-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cffee45a1724600a728967819b23e80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1adfdb2abb71494bbc824f175a501cd3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:50.469102 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d48db2-0a04-4595-b4d5-e24114faea79 tempest-VolumeQuotaClassesTest-1873665499 tempest-VolumeQuotaClassesTest-1873665499-project-admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:50.469388 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d48db2-0a04-4595-b4d5-e24114faea79 tempest-VolumeQuotaClassesTest-1873665499 tempest-VolumeQuotaClassesTest-1873665499-project-admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:50.469670 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d48db2-0a04-4595-b4d5-e24114faea79 tempest-VolumeQuotaClassesTest-1873665499 tempest-VolumeQuotaClassesTest-1873665499-project-admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:50.470405 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d48db2-0a04-4595-b4d5-e24114faea79 tempest-VolumeQuotaClassesTest-1873665499 tempest-VolumeQuotaClassesTest-1873665499-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:55:50.470719 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d48db2-0a04-4595-b4d5-e24114faea79 tempest-VolumeQuotaClassesTest-1873665499 tempest-VolumeQuotaClassesTest-1873665499-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:50.472382 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d48db2-0a04-4595-b4d5-e24114faea79 tempest-VolumeQuotaClassesTest-1873665499 tempest-VolumeQuotaClassesTest-1873665499-project-admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:50.484579 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1252/2486] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 2 19:55:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:55:50.488853 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6a9c26-ab48-40c0-9530-9428dd094735 service cinder] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:50.498117 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6a9c26-ab48-40c0-9530-9428dd094735 service cinder] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:50.502705 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f97edb-104b-4e63-aa7d-92c8c59b54e9 tempest-VolumeQuotaClassesTest-1873665499 tempest-VolumeQuotaClassesTest-1873665499-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cffee45a1724600a728967819b23e80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1adfdb2abb71494bbc824f175a501cd3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:50.503430 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f97edb-104b-4e63-aa7d-92c8c59b54e9 tempest-VolumeQuotaClassesTest-1873665499 tempest-VolumeQuotaClassesTest-1873665499-project-admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:50.503718 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f97edb-104b-4e63-aa7d-92c8c59b54e9 tempest-VolumeQuotaClassesTest-1873665499 tempest-VolumeQuotaClassesTest-1873665499-project-admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:50.504005 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f97edb-104b-4e63-aa7d-92c8c59b54e9 tempest-VolumeQuotaClassesTest-1873665499 tempest-VolumeQuotaClassesTest-1873665499-project-admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:50.504725 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f97edb-104b-4e63-aa7d-92c8c59b54e9 tempest-VolumeQuotaClassesTest-1873665499 tempest-VolumeQuotaClassesTest-1873665499-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:50.506720 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f97edb-104b-4e63-aa7d-92c8c59b54e9 tempest-VolumeQuotaClassesTest-1873665499 tempest-VolumeQuotaClassesTest-1873665499-project-admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:50.581110 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1235/2487] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:55:50 2022] POST /identity/v3/projects => generated 355 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 02 19:55:50.584929 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d48db2-0a04-4595-b4d5-e24114faea79 tempest-VolumeQuotaClassesTest-1873665499 tempest-VolumeQuotaClassesTest-1873665499-project-admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:50.595415 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d48db2-0a04-4595-b4d5-e24114faea79 tempest-VolumeQuotaClassesTest-1873665499 tempest-VolumeQuotaClassesTest-1873665499-project-admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:50.599924 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:55:50.651166 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4d59d4-4770-4f3d-bb2f-39cddc138988 tempest-VolumeQuotaClassesTest-1873665499 tempest-VolumeQuotaClassesTest-1873665499-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cffee45a1724600a728967819b23e80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1adfdb2abb71494bbc824f175a501cd3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:50.651824 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d59d4-4770-4f3d-bb2f-39cddc138988 tempest-VolumeQuotaClassesTest-1873665499 tempest-VolumeQuotaClassesTest-1873665499-project-admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:50.652117 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d59d4-4770-4f3d-bb2f-39cddc138988 tempest-VolumeQuotaClassesTest-1873665499 tempest-VolumeQuotaClassesTest-1873665499-project-admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:50.652401 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d59d4-4770-4f3d-bb2f-39cddc138988 tempest-VolumeQuotaClassesTest-1873665499 tempest-VolumeQuotaClassesTest-1873665499-project-admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:50.653134 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4d59d4-4770-4f3d-bb2f-39cddc138988 tempest-VolumeQuotaClassesTest-1873665499 tempest-VolumeQuotaClassesTest-1873665499-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:55:50.653446 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4d59d4-4770-4f3d-bb2f-39cddc138988 tempest-VolumeQuotaClassesTest-1873665499 tempest-VolumeQuotaClassesTest-1873665499-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:50.655098 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4d59d4-4770-4f3d-bb2f-39cddc138988 tempest-VolumeQuotaClassesTest-1873665499 tempest-VolumeQuotaClassesTest-1873665499-project-admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:50.669687 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1253/2488] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 2 19:55:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:55:50.703162 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f97edb-104b-4e63-aa7d-92c8c59b54e9 tempest-VolumeQuotaClassesTest-1873665499 tempest-VolumeQuotaClassesTest-1873665499-project-admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:50.714096 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f97edb-104b-4e63-aa7d-92c8c59b54e9 tempest-VolumeQuotaClassesTest-1873665499 tempest-VolumeQuotaClassesTest-1873665499-project-admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:50.719234 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f889be-322d-4351-bff0-83efed1557ff tempest-VolumeQuotaClassesTest-1873665499 tempest-VolumeQuotaClassesTest-1873665499-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cffee45a1724600a728967819b23e80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1adfdb2abb71494bbc824f175a501cd3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:50.720094 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f889be-322d-4351-bff0-83efed1557ff tempest-VolumeQuotaClassesTest-1873665499 tempest-VolumeQuotaClassesTest-1873665499-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:50.720386 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f889be-322d-4351-bff0-83efed1557ff tempest-VolumeQuotaClassesTest-1873665499 tempest-VolumeQuotaClassesTest-1873665499-project-admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:50.720670 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f889be-322d-4351-bff0-83efed1557ff tempest-VolumeQuotaClassesTest-1873665499 tempest-VolumeQuotaClassesTest-1873665499-project-admin] PATH_INFO: `/v3/projects/201cd0b25d5348b6aff805330425c867` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:50.721855 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f889be-322d-4351-bff0-83efed1557ff tempest-VolumeQuotaClassesTest-1873665499 tempest-VolumeQuotaClassesTest-1873665499-project-admin] RBAC: Authorizing `identity:delete_project(project_id=201cd0b25d5348b6aff805330425c867)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:50.723790 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f889be-322d-4351-bff0-83efed1557ff tempest-VolumeQuotaClassesTest-1873665499 tempest-VolumeQuotaClassesTest-1873665499-project-admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:50.839919 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-72f889be-322d-4351-bff0-83efed1557ff tempest-VolumeQuotaClassesTest-1873665499 tempest-VolumeQuotaClassesTest-1873665499-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '201cd0b25d5348b6aff805330425c867', 'request_id': 'req-72f889be-322d-4351-bff0-83efed1557ff'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:50.901515 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-72f889be-322d-4351-bff0-83efed1557ff tempest-VolumeQuotaClassesTest-1873665499 tempest-VolumeQuotaClassesTest-1873665499-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '201cd0b25d5348b6aff805330425c867', 'request_id': 'req-72f889be-322d-4351-bff0-83efed1557ff'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:50.907943 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-72f889be-322d-4351-bff0-83efed1557ff tempest-VolumeQuotaClassesTest-1873665499 tempest-VolumeQuotaClassesTest-1873665499-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '201cd0b25d5348b6aff805330425c867', 'request_id': 'req-72f889be-322d-4351-bff0-83efed1557ff'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:50.919911 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-72f889be-322d-4351-bff0-83efed1557ff tempest-VolumeQuotaClassesTest-1873665499 tempest-VolumeQuotaClassesTest-1873665499-project-admin] The token cache is being invalidate because project 201cd0b25d5348b6aff805330425c867 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:55:50.920247 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-72f889be-322d-4351-bff0-83efed1557ff tempest-VolumeQuotaClassesTest-1873665499 tempest-VolumeQuotaClassesTest-1873665499-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:50.920969 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-72f889be-322d-4351-bff0-83efed1557ff tempest-VolumeQuotaClassesTest-1873665499 tempest-VolumeQuotaClassesTest-1873665499-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:50.922656 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1236/2489] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:55:50 2022] DELETE /identity/v3/projects/201cd0b25d5348b6aff805330425c867 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:55:51.591853 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4d59d4-4770-4f3d-bb2f-39cddc138988 tempest-VolumeQuotaClassesTest-1873665499 tempest-VolumeQuotaClassesTest-1873665499-project-admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:51.593832 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-dd4d59d4-4770-4f3d-bb2f-39cddc138988 tempest-VolumeQuotaClassesTest-1873665499 tempest-VolumeQuotaClassesTest-1873665499-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:51.665896 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4d59d4-4770-4f3d-bb2f-39cddc138988 tempest-VolumeQuotaClassesTest-1873665499 tempest-VolumeQuotaClassesTest-1873665499-project-admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:51.723194 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effc1c05-2971-417b-a0f0-19754f39b46d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:51.724130 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc1c05-2971-417b-a0f0-19754f39b46d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:51.724517 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc1c05-2971-417b-a0f0-19754f39b46d admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:51.724797 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc1c05-2971-417b-a0f0-19754f39b46d admin admin] PATH_INFO: `/v3/users/2cde22927b234b20b137629931aaaa8f` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:51.725977 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effc1c05-2971-417b-a0f0-19754f39b46d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cde22927b234b20b137629931aaaa8f)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:51.728624 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effc1c05-2971-417b-a0f0-19754f39b46d admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:51.913334 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-effc1c05-2971-417b-a0f0-19754f39b46d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cde22927b234b20b137629931aaaa8f'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:51.918109 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:55:51.921043 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-effc1c05-2971-417b-a0f0-19754f39b46d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cde22927b234b20b137629931aaaa8f'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:51.998414 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-effc1c05-2971-417b-a0f0-19754f39b46d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cde22927b234b20b137629931aaaa8f'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:51.999129 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-effc1c05-2971-417b-a0f0-19754f39b46d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cde22927b234b20b137629931aaaa8f'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:51.999801 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-effc1c05-2971-417b-a0f0-19754f39b46d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cde22927b234b20b137629931aaaa8f'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:52.017921 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1254/2490] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:55:51 2022] DELETE /identity/v3/users/2cde22927b234b20b137629931aaaa8f => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:55:52.022623 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca4da6-a3b8-456c-b544-05a3a125e1e5 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:52.023113 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca4da6-a3b8-456c-b544-05a3a125e1e5 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:52.023248 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca4da6-a3b8-456c-b544-05a3a125e1e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:52.040872 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-9dca4da6-a3b8-456c-b544-05a3a125e1e5 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:55:52.042090 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-9dca4da6-a3b8-456c-b544-05a3a125e1e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:52.051388 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:INSERT=7,keystone:SELECT=86,keystone:DELETE=7 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:55:52.099687 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1237/2491] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:55:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:55:52.109677 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effc1c05-2971-417b-a0f0-19754f39b46d admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:52.111112 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-effc1c05-2971-417b-a0f0-19754f39b46d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:52.169692 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effc1c05-2971-417b-a0f0-19754f39b46d admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:52.173302 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a4deae-4e55-4526-b481-6be367bb98f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:52.173945 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a4deae-4e55-4526-b481-6be367bb98f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:52.174232 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a4deae-4e55-4526-b481-6be367bb98f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:52.174504 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a4deae-4e55-4526-b481-6be367bb98f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:52.183952 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a4deae-4e55-4526-b481-6be367bb98f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:52.187656 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a4deae-4e55-4526-b481-6be367bb98f9 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:52.194631 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1255/2492] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:55:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:55:52.582554 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca4da6-a3b8-456c-b544-05a3a125e1e5 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:52.584216 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-9dca4da6-a3b8-456c-b544-05a3a125e1e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:52.606175 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca4da6-a3b8-456c-b544-05a3a125e1e5 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:52.606399 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a4deae-4e55-4526-b481-6be367bb98f9 service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:52.608197 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-12a4deae-4e55-4526-b481-6be367bb98f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:52.622434 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b11d80-f16f-4012-ad22-51742b5a7446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:52.623561 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b11d80-f16f-4012-ad22-51742b5a7446 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:52.624011 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b11d80-f16f-4012-ad22-51742b5a7446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:52.624486 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b11d80-f16f-4012-ad22-51742b5a7446 admin admin] PATH_INFO: `/v3/projects/f0817ef18e0b491e8518e1a42dcfed97` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:52.625909 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b11d80-f16f-4012-ad22-51742b5a7446 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0817ef18e0b491e8518e1a42dcfed97)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:52.627956 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b11d80-f16f-4012-ad22-51742b5a7446 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:52.668141 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a4deae-4e55-4526-b481-6be367bb98f9 service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:52.671775 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64af3ec4-7d86-4342-b198-09217e7d78f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:52.672423 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af3ec4-7d86-4342-b198-09217e7d78f2 service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:52.672710 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af3ec4-7d86-4342-b198-09217e7d78f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:52.672994 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af3ec4-7d86-4342-b198-09217e7d78f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:52.674790 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-64af3ec4-7d86-4342-b198-09217e7d78f2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:52.723237 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-c7b11d80-f16f-4012-ad22-51742b5a7446 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0817ef18e0b491e8518e1a42dcfed97', 'request_id': 'req-c7b11d80-f16f-4012-ad22-51742b5a7446'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:52.737722 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64af3ec4-7d86-4342-b198-09217e7d78f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:52.739205 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64af3ec4-7d86-4342-b198-09217e7d78f2 service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:52.748799 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1256/2493] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:55:52 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:55:52.802671 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-c7b11d80-f16f-4012-ad22-51742b5a7446 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0817ef18e0b491e8518e1a42dcfed97', 'request_id': 'req-c7b11d80-f16f-4012-ad22-51742b5a7446'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:52.810375 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-c7b11d80-f16f-4012-ad22-51742b5a7446 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0817ef18e0b491e8518e1a42dcfed97', 'request_id': 'req-c7b11d80-f16f-4012-ad22-51742b5a7446'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:52.824801 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-c7b11d80-f16f-4012-ad22-51742b5a7446 admin admin] The token cache is being invalidate because project f0817ef18e0b491e8518e1a42dcfed97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:55:52.825120 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-c7b11d80-f16f-4012-ad22-51742b5a7446 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:52.825763 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-c7b11d80-f16f-4012-ad22-51742b5a7446 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:52.827525 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1238/2494] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:55:52 2022] DELETE /identity/v3/projects/f0817ef18e0b491e8518e1a42dcfed97 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:55:52.831507 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64af3ec4-7d86-4342-b198-09217e7d78f2 service cinder] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:52.842104 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64af3ec4-7d86-4342-b198-09217e7d78f2 service cinder] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:52.861162 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:55:52.900726 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681e897f-951b-4dd5-b241-9fe2276e4533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:52.901602 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e897f-951b-4dd5-b241-9fe2276e4533 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:52.901883 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e897f-951b-4dd5-b241-9fe2276e4533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:52.902158 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e897f-951b-4dd5-b241-9fe2276e4533 admin admin] PATH_INFO: `/v3/users/5cffee45a1724600a728967819b23e80` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:52.903412 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681e897f-951b-4dd5-b241-9fe2276e4533 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cffee45a1724600a728967819b23e80)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:52.905393 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681e897f-951b-4dd5-b241-9fe2276e4533 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:53.086064 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-681e897f-951b-4dd5-b241-9fe2276e4533 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cffee45a1724600a728967819b23e80'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:53.093089 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-681e897f-951b-4dd5-b241-9fe2276e4533 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cffee45a1724600a728967819b23e80'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:53.175713 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-681e897f-951b-4dd5-b241-9fe2276e4533 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cffee45a1724600a728967819b23e80'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:53.176496 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-681e897f-951b-4dd5-b241-9fe2276e4533 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cffee45a1724600a728967819b23e80'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:53.177185 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-681e897f-951b-4dd5-b241-9fe2276e4533 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cffee45a1724600a728967819b23e80'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:53.195030 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1257/2495] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:55:52 2022] DELETE /identity/v3/users/5cffee45a1724600a728967819b23e80 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:55:53.658017 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b11d80-f16f-4012-ad22-51742b5a7446 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:53.658547 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-c7b11d80-f16f-4012-ad22-51742b5a7446 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:53.671086 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b11d80-f16f-4012-ad22-51742b5a7446 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:53.727968 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69014dc7-d50e-466f-971a-e65c51304ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:53.728824 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69014dc7-d50e-466f-971a-e65c51304ffb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:53.729106 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69014dc7-d50e-466f-971a-e65c51304ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:53.729379 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69014dc7-d50e-466f-971a-e65c51304ffb admin admin] PATH_INFO: `/v3/projects/1adfdb2abb71494bbc824f175a501cd3` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:53.730714 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69014dc7-d50e-466f-971a-e65c51304ffb admin admin] RBAC: Authorizing `identity:delete_project(project_id=1adfdb2abb71494bbc824f175a501cd3)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:53.732642 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69014dc7-d50e-466f-971a-e65c51304ffb admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:53.825776 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:55:53.832165 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-69014dc7-d50e-466f-971a-e65c51304ffb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1adfdb2abb71494bbc824f175a501cd3', 'request_id': 'req-69014dc7-d50e-466f-971a-e65c51304ffb'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:53.911746 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-69014dc7-d50e-466f-971a-e65c51304ffb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1adfdb2abb71494bbc824f175a501cd3', 'request_id': 'req-69014dc7-d50e-466f-971a-e65c51304ffb'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:53.917964 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-69014dc7-d50e-466f-971a-e65c51304ffb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1adfdb2abb71494bbc824f175a501cd3', 'request_id': 'req-69014dc7-d50e-466f-971a-e65c51304ffb'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:53.929544 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-69014dc7-d50e-466f-971a-e65c51304ffb admin admin] The token cache is being invalidate because project 1adfdb2abb71494bbc824f175a501cd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:55:53.929826 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-69014dc7-d50e-466f-971a-e65c51304ffb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:53.930451 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-69014dc7-d50e-466f-971a-e65c51304ffb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:53.932615 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1239/2496] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:55:53 2022] DELETE /identity/v3/projects/1adfdb2abb71494bbc824f175a501cd3 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:55:53.937432 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d8f918-e22c-4e42-8f73-288216d48f83 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:53.937894 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d8f918-e22c-4e42-8f73-288216d48f83 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:53.937982 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d8f918-e22c-4e42-8f73-288216d48f83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:53.956351 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-c1d8f918-e22c-4e42-8f73-288216d48f83 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:55:53.957663 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-c1d8f918-e22c-4e42-8f73-288216d48f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:54.017030 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1258/2497] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:55:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:55:54.024708 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614a6c56-25b4-4406-8326-0af1a2461c4e None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:54.024996 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614a6c56-25b4-4406-8326-0af1a2461c4e None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:54.025936 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614a6c56-25b4-4406-8326-0af1a2461c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:54.044157 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-614a6c56-25b4-4406-8326-0af1a2461c4e None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:55:54.045362 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-614a6c56-25b4-4406-8326-0af1a2461c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:54.053066 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1240/2498] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:55:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:55:54.057322 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d8f918-e22c-4e42-8f73-288216d48f83 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:54.067105 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d8f918-e22c-4e42-8f73-288216d48f83 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:54.071763 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665924e3-bc31-44cb-8cfc-a26404f9191f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:54.072402 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665924e3-bc31-44cb-8cfc-a26404f9191f admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:54.072692 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665924e3-bc31-44cb-8cfc-a26404f9191f admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:54.072974 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665924e3-bc31-44cb-8cfc-a26404f9191f admin admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:54.073736 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665924e3-bc31-44cb-8cfc-a26404f9191f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:55:54.074049 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665924e3-bc31-44cb-8cfc-a26404f9191f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:54.075708 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665924e3-bc31-44cb-8cfc-a26404f9191f admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:54.087222 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1259/2499] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 2 19:55:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:55:54.091911 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614a6c56-25b4-4406-8326-0af1a2461c4e None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:54.100995 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614a6c56-25b4-4406-8326-0af1a2461c4e None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:54.106180 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e800d7b-08c3-4a79-a36b-bc17f6668b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:54.106918 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e800d7b-08c3-4a79-a36b-bc17f6668b7f admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:54.107200 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e800d7b-08c3-4a79-a36b-bc17f6668b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:54.107473 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e800d7b-08c3-4a79-a36b-bc17f6668b7f admin admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:54.108158 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e800d7b-08c3-4a79-a36b-bc17f6668b7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:54.110016 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e800d7b-08c3-4a79-a36b-bc17f6668b7f admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:54.171083 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1241/2500] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:55:54 2022] POST /identity/v3/projects => generated 387 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:55:54.175020 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665924e3-bc31-44cb-8cfc-a26404f9191f admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:54.185131 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665924e3-bc31-44cb-8cfc-a26404f9191f admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:54.211662 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:55:54.244690 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5d977c-dab9-4833-aba2-31ea8152ea00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:54.245358 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d977c-dab9-4833-aba2-31ea8152ea00 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:54.245646 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d977c-dab9-4833-aba2-31ea8152ea00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:54.245922 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d977c-dab9-4833-aba2-31ea8152ea00 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:54.246611 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5d977c-dab9-4833-aba2-31ea8152ea00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:54.248598 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5d977c-dab9-4833-aba2-31ea8152ea00 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:54.317739 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1260/2501] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:55:54 2022] POST /identity/v3/users => generated 294 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:55:54.322631 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e800d7b-08c3-4a79-a36b-bc17f6668b7f admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:54.333702 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e800d7b-08c3-4a79-a36b-bc17f6668b7f admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:54.339101 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de7c18a-8ebb-4ab5-80e5-3ce4d4df595e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:54.339771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de7c18a-8ebb-4ab5-80e5-3ce4d4df595e admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:54.340061 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de7c18a-8ebb-4ab5-80e5-3ce4d4df595e admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:54.340336 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de7c18a-8ebb-4ab5-80e5-3ce4d4df595e admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:54.341078 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de7c18a-8ebb-4ab5-80e5-3ce4d4df595e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:54.342719 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de7c18a-8ebb-4ab5-80e5-3ce4d4df595e admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:54.353444 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1242/2502] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:55:54.357534 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5d977c-dab9-4833-aba2-31ea8152ea00 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:54.367727 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5d977c-dab9-4833-aba2-31ea8152ea00 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:54.372771 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37d2882-1c78-417c-847f-528614328fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:54.373546 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37d2882-1c78-417c-847f-528614328fd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:54.373744 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37d2882-1c78-417c-847f-528614328fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:54.374023 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37d2882-1c78-417c-847f-528614328fd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:54.374784 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37d2882-1c78-417c-847f-528614328fd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:54.376427 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37d2882-1c78-417c-847f-528614328fd1 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:54.387590 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1261/2503] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:55:54.392038 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de7c18a-8ebb-4ab5-80e5-3ce4d4df595e admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:54.402526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de7c18a-8ebb-4ab5-80e5-3ce4d4df595e admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:54.408094 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b99d695-be97-4e8a-bfeb-7e69f1b6442b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:54.409098 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b99d695-be97-4e8a-bfeb-7e69f1b6442b admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:54.409454 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b99d695-be97-4e8a-bfeb-7e69f1b6442b admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:54.409736 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b99d695-be97-4e8a-bfeb-7e69f1b6442b admin admin] PATH_INFO: `/v3/projects/a6dfa1fcf1ad4fde94627804a8b87ea7/users/5575aa83e93f491c88a7d1cf83e8d7d8/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:54.424678 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b99d695-be97-4e8a-bfeb-7e69f1b6442b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6dfa1fcf1ad4fde94627804a8b87ea7, user_id=5575aa83e93f491c88a7d1cf83e8d7d8, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:54.428029 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b99d695-be97-4e8a-bfeb-7e69f1b6442b admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:54.495801 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1243/2504] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:55:54 2022] PUT /identity/v3/projects/a6dfa1fcf1ad4fde94627804a8b87ea7/users/5575aa83e93f491c88a7d1cf83e8d7d8/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:55:54.509129 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343b11e3-5716-4d39-b012-f0a90a0cb8d5 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:54.509426 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343b11e3-5716-4d39-b012-f0a90a0cb8d5 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:54.510239 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343b11e3-5716-4d39-b012-f0a90a0cb8d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:54.538194 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-343b11e3-5716-4d39-b012-f0a90a0cb8d5 None None] MFA Rules not processed for user `5575aa83e93f491c88a7d1cf83e8d7d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:55:54.539573 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-343b11e3-5716-4d39-b012-f0a90a0cb8d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:54.608696 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1262/2505] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:55:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 19:55:54.613409 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b99d695-be97-4e8a-bfeb-7e69f1b6442b admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:54.678041 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b99d695-be97-4e8a-bfeb-7e69f1b6442b admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:54.683562 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8e2b0a-e257-4a1c-b056-32757f5d3b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:54.684233 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e2b0a-e257-4a1c-b056-32757f5d3b84 admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:54.685958 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e2b0a-e257-4a1c-b056-32757f5d3b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:54.685958 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e2b0a-e257-4a1c-b056-32757f5d3b84 admin admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:54.685958 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8e2b0a-e257-4a1c-b056-32757f5d3b84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:54.687621 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8e2b0a-e257-4a1c-b056-32757f5d3b84 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:54.758911 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1244/2506] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:55:54 2022] POST /identity/v3/projects => generated 387 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:55:54.763684 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343b11e3-5716-4d39-b012-f0a90a0cb8d5 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:54.775977 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343b11e3-5716-4d39-b012-f0a90a0cb8d5 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:54.833976 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cc16f3-5fe1-451e-a70f-e0ac3e4eb0fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:54.834659 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cc16f3-5fe1-451e-a70f-e0ac3e4eb0fe admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:54.835134 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cc16f3-5fe1-451e-a70f-e0ac3e4eb0fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:54.835314 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cc16f3-5fe1-451e-a70f-e0ac3e4eb0fe admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:54.836026 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cc16f3-5fe1-451e-a70f-e0ac3e4eb0fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:54.838478 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cc16f3-5fe1-451e-a70f-e0ac3e4eb0fe admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:54.915567 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1263/2507] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:55:54 2022] POST /identity/v3/users => generated 294 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:55:54.916953 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8e2b0a-e257-4a1c-b056-32757f5d3b84 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:54.927485 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8e2b0a-e257-4a1c-b056-32757f5d3b84 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:54.932718 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f8a2b4-4174-4aeb-9a88-40b81ab63a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:54.933382 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f8a2b4-4174-4aeb-9a88-40b81ab63a88 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:54.933668 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f8a2b4-4174-4aeb-9a88-40b81ab63a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:54.933957 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f8a2b4-4174-4aeb-9a88-40b81ab63a88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:54.934678 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f8a2b4-4174-4aeb-9a88-40b81ab63a88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:54.936318 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f8a2b4-4174-4aeb-9a88-40b81ab63a88 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:54.950330 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1245/2508] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:55:54.952069 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cc16f3-5fe1-451e-a70f-e0ac3e4eb0fe admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:54.962340 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cc16f3-5fe1-451e-a70f-e0ac3e4eb0fe admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:54.967154 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a90ca7a-7de8-45a8-adbc-abf59ca36cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:54.967813 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a90ca7a-7de8-45a8-adbc-abf59ca36cd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:54.968103 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a90ca7a-7de8-45a8-adbc-abf59ca36cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:54.968375 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a90ca7a-7de8-45a8-adbc-abf59ca36cd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:54.969164 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a90ca7a-7de8-45a8-adbc-abf59ca36cd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:54.970899 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a90ca7a-7de8-45a8-adbc-abf59ca36cd5 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:54.980942 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1264/2509] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:55:54.985259 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f8a2b4-4174-4aeb-9a88-40b81ab63a88 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:54.995691 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f8a2b4-4174-4aeb-9a88-40b81ab63a88 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:55.000319 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e271d120-e7d9-41f4-9436-7c621ba05c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:55.001190 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e271d120-e7d9-41f4-9436-7c621ba05c8c admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:55.001472 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e271d120-e7d9-41f4-9436-7c621ba05c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:55.001746 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e271d120-e7d9-41f4-9436-7c621ba05c8c admin admin] PATH_INFO: `/v3/projects/5d12fe55754f46ff84edef8863bea044/users/cdfe8b0a0cee46b186022b0384ea66ea/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:55.017080 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e271d120-e7d9-41f4-9436-7c621ba05c8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d12fe55754f46ff84edef8863bea044, user_id=cdfe8b0a0cee46b186022b0384ea66ea, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:55.019748 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e271d120-e7d9-41f4-9436-7c621ba05c8c admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:55.082569 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1246/2510] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:55:54 2022] PUT /identity/v3/projects/5d12fe55754f46ff84edef8863bea044/users/cdfe8b0a0cee46b186022b0384ea66ea/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 02 19:55:55.093456 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc46e58-b8cc-4baf-b84c-05f6a96ac40b None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:55.093777 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc46e58-b8cc-4baf-b84c-05f6a96ac40b None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:55.094053 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc46e58-b8cc-4baf-b84c-05f6a96ac40b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:55.121872 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-9bc46e58-b8cc-4baf-b84c-05f6a96ac40b None None] MFA Rules not processed for user `cdfe8b0a0cee46b186022b0384ea66ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:55:55.123124 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-9bc46e58-b8cc-4baf-b84c-05f6a96ac40b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:55.127265 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:55:55.188976 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1265/2511] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:55:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:55:55.193584 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e271d120-e7d9-41f4-9436-7c621ba05c8c admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:55.211406 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:55:55.254430 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e271d120-e7d9-41f4-9436-7c621ba05c8c admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:55.259182 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23755a7-a05d-4941-8936-16c421749c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:55.259881 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23755a7-a05d-4941-8936-16c421749c0f admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:55.260168 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23755a7-a05d-4941-8936-16c421749c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:55.260448 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23755a7-a05d-4941-8936-16c421749c0f admin admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:55.261137 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23755a7-a05d-4941-8936-16c421749c0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:55.263066 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23755a7-a05d-4941-8936-16c421749c0f admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:55.371198 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1247/2512] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:55:55 2022] POST /identity/v3/projects => generated 387 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:55:55.375391 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc46e58-b8cc-4baf-b84c-05f6a96ac40b None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:55.386072 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc46e58-b8cc-4baf-b84c-05f6a96ac40b None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:55.442498 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3971b4-7709-40af-ab15-c4386b74f3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:55.443161 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3971b4-7709-40af-ab15-c4386b74f3f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:55.443451 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3971b4-7709-40af-ab15-c4386b74f3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:55.443734 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3971b4-7709-40af-ab15-c4386b74f3f6 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:55.444506 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3971b4-7709-40af-ab15-c4386b74f3f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:55.446471 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3971b4-7709-40af-ab15-c4386b74f3f6 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:55.545813 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1266/2513] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:55:55 2022] POST /identity/v3/users => generated 300 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:55:55.549875 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23755a7-a05d-4941-8936-16c421749c0f admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:55.560830 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23755a7-a05d-4941-8936-16c421749c0f admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:55.567556 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8731244b-c95f-4768-b183-431e83d20272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:55.568218 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731244b-c95f-4768-b183-431e83d20272 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:55.568703 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731244b-c95f-4768-b183-431e83d20272 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:55.569116 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731244b-c95f-4768-b183-431e83d20272 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:55.569864 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8731244b-c95f-4768-b183-431e83d20272 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:55.571507 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8731244b-c95f-4768-b183-431e83d20272 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:55.583233 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1248/2514] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:55:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:55:55.587422 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3971b4-7709-40af-ab15-c4386b74f3f6 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:55.597193 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3971b4-7709-40af-ab15-c4386b74f3f6 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:55.601738 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c96329-cb4f-4105-8ce1-5da600bd2354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:55.602632 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c96329-cb4f-4105-8ce1-5da600bd2354 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:55.602951 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c96329-cb4f-4105-8ce1-5da600bd2354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:55.603231 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c96329-cb4f-4105-8ce1-5da600bd2354 admin admin] PATH_INFO: `/v3/projects/e91a30768c2440488f501283f11121ed/users/a8a20bcf1ca34a9093608ab426305dbf/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:55.617414 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c96329-cb4f-4105-8ce1-5da600bd2354 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e91a30768c2440488f501283f11121ed, user_id=a8a20bcf1ca34a9093608ab426305dbf, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:55.620587 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c96329-cb4f-4105-8ce1-5da600bd2354 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:55.672875 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1267/2515] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:55:55 2022] PUT /identity/v3/projects/e91a30768c2440488f501283f11121ed/users/a8a20bcf1ca34a9093608ab426305dbf/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:55:55.676602 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8731244b-c95f-4768-b183-431e83d20272 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:55.686918 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8731244b-c95f-4768-b183-431e83d20272 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:55.742489 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9d0fe5-60f2-4848-966e-b95eb0c3392e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:55.743195 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d0fe5-60f2-4848-966e-b95eb0c3392e admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:55.743489 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d0fe5-60f2-4848-966e-b95eb0c3392e admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:55.743802 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d0fe5-60f2-4848-966e-b95eb0c3392e admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:55.744531 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9d0fe5-60f2-4848-966e-b95eb0c3392e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:55.746124 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9d0fe5-60f2-4848-966e-b95eb0c3392e admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:55.755745 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1249/2516] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:55:55 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:55:55.759849 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c96329-cb4f-4105-8ce1-5da600bd2354 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:55.770206 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c96329-cb4f-4105-8ce1-5da600bd2354 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:55.774735 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66043c9-a7cc-414e-8349-ac64266ebb0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:55.776323 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66043c9-a7cc-414e-8349-ac64266ebb0a admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:55.776610 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66043c9-a7cc-414e-8349-ac64266ebb0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:55.776914 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66043c9-a7cc-414e-8349-ac64266ebb0a admin admin] PATH_INFO: `/v3/projects/e91a30768c2440488f501283f11121ed/users/a8a20bcf1ca34a9093608ab426305dbf/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:55.779015 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66043c9-a7cc-414e-8349-ac64266ebb0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e91a30768c2440488f501283f11121ed, user_id=a8a20bcf1ca34a9093608ab426305dbf, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:55.781656 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66043c9-a7cc-414e-8349-ac64266ebb0a admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:55.792550 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1268/2517] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:55:55 2022] PUT /identity/v3/projects/e91a30768c2440488f501283f11121ed/users/a8a20bcf1ca34a9093608ab426305dbf/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:55:55.802680 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1ec1c-d964-4590-8de7-2d0640c4d2a3 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:55.803010 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1ec1c-d964-4590-8de7-2d0640c4d2a3 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:55.803296 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1ec1c-d964-4590-8de7-2d0640c4d2a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:55.830666 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-8cc1ec1c-d964-4590-8de7-2d0640c4d2a3 None None] MFA Rules not processed for user `a8a20bcf1ca34a9093608ab426305dbf`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:55:55.831905 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-8cc1ec1c-d964-4590-8de7-2d0640c4d2a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:55.898861 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1250/2518] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:55:55 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:55:55.909979 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66043c9-a7cc-414e-8349-ac64266ebb0a admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:55.911830 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-e66043c9-a7cc-414e-8349-ac64266ebb0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:55.969620 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66043c9-a7cc-414e-8349-ac64266ebb0a admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:55.973259 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03d5c87-552d-423d-8f1c-f75078275d50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:55.979602 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03d5c87-552d-423d-8f1c-f75078275d50 service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:55.979602 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03d5c87-552d-423d-8f1c-f75078275d50 service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:55.979602 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03d5c87-552d-423d-8f1c-f75078275d50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:55.984353 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03d5c87-552d-423d-8f1c-f75078275d50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:55.985805 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03d5c87-552d-423d-8f1c-f75078275d50 service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:55.991075 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:55:55.996594 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1269/2519] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:55:55 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:55:57.049665 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc1ec1c-d964-4590-8de7-2d0640c4d2a3 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:57.051112 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-8cc1ec1c-d964-4590-8de7-2d0640c4d2a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:57.063213 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc1ec1c-d964-4590-8de7-2d0640c4d2a3 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:57.066651 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03766908-b002-4a65-9517-ec95a19c3cb8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:57.067334 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03766908-b002-4a65-9517-ec95a19c3cb8 service cinder] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:57.067619 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03766908-b002-4a65-9517-ec95a19c3cb8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:57.067892 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03766908-b002-4a65-9517-ec95a19c3cb8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:57.076448 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:55:57.124110 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03766908-b002-4a65-9517-ec95a19c3cb8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:57.125586 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03766908-b002-4a65-9517-ec95a19c3cb8 service cinder] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:57.135286 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1251/2520] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:55:57 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:55:58.461845 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03d5c87-552d-423d-8f1c-f75078275d50 service cinder] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:58.463456 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-b03d5c87-552d-423d-8f1c-f75078275d50 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:58.527530 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03d5c87-552d-423d-8f1c-f75078275d50 service cinder] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:58.532080 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4a821a-5459-4f76-86be-2cca633a0e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:58.532945 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4a821a-5459-4f76-86be-2cca633a0e91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:58.533235 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4a821a-5459-4f76-86be-2cca633a0e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:58.533512 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4a821a-5459-4f76-86be-2cca633a0e91 admin admin] PATH_INFO: `/v3/users/2a838cff9e1c4eedb2cb7fe71b2fbfbe` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:58.534688 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4a821a-5459-4f76-86be-2cca633a0e91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a838cff9e1c4eedb2cb7fe71b2fbfbe)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:58.536639 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4a821a-5459-4f76-86be-2cca633a0e91 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:58.690305 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-3f4a821a-5459-4f76-86be-2cca633a0e91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a838cff9e1c4eedb2cb7fe71b2fbfbe'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:58.697893 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-3f4a821a-5459-4f76-86be-2cca633a0e91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a838cff9e1c4eedb2cb7fe71b2fbfbe'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:58.773876 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-3f4a821a-5459-4f76-86be-2cca633a0e91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a838cff9e1c4eedb2cb7fe71b2fbfbe'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:58.774525 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-3f4a821a-5459-4f76-86be-2cca633a0e91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a838cff9e1c4eedb2cb7fe71b2fbfbe'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:58.775160 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-3f4a821a-5459-4f76-86be-2cca633a0e91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a838cff9e1c4eedb2cb7fe71b2fbfbe'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:58.795790 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1270/2521] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:55:58 2022] DELETE /identity/v3/users/2a838cff9e1c4eedb2cb7fe71b2fbfbe => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:55:58.800763 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39bdb67-8b00-428d-8a23-97e47ab2f8b4 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:58.800763 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39bdb67-8b00-428d-8a23-97e47ab2f8b4 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:58.801393 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39bdb67-8b00-428d-8a23-97e47ab2f8b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:58.822662 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-b39bdb67-8b00-428d-8a23-97e47ab2f8b4 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:55:58.824149 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-b39bdb67-8b00-428d-8a23-97e47ab2f8b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:58.891023 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1252/2522] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:55:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 19:55:58.900826 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4a821a-5459-4f76-86be-2cca633a0e91 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:58.902269 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-3f4a821a-5459-4f76-86be-2cca633a0e91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:58.960320 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4a821a-5459-4f76-86be-2cca633a0e91 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:58.964639 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e39aa2b-f966-4a9a-90a9-2fc4962fdc40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:58.965288 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e39aa2b-f966-4a9a-90a9-2fc4962fdc40 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:58.965572 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e39aa2b-f966-4a9a-90a9-2fc4962fdc40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:58.965847 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e39aa2b-f966-4a9a-90a9-2fc4962fdc40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:58.975008 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e39aa2b-f966-4a9a-90a9-2fc4962fdc40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:58.976444 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e39aa2b-f966-4a9a-90a9-2fc4962fdc40 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:58.985911 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1271/2523] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:55:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:55:59.381167 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39bdb67-8b00-428d-8a23-97e47ab2f8b4 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:59.382592 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-b39bdb67-8b00-428d-8a23-97e47ab2f8b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:55:59.397432 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39bdb67-8b00-428d-8a23-97e47ab2f8b4 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:59.402190 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69d3c85-ca68-4042-bc5f-ada7ea6fc9ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:59.403209 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69d3c85-ca68-4042-bc5f-ada7ea6fc9ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:59.403434 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69d3c85-ca68-4042-bc5f-ada7ea6fc9ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:59.403711 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69d3c85-ca68-4042-bc5f-ada7ea6fc9ac admin admin] PATH_INFO: `/v3/projects/d1cfed5b12c0402ba50f1a7ad230e891` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:59.405619 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69d3c85-ca68-4042-bc5f-ada7ea6fc9ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1cfed5b12c0402ba50f1a7ad230e891)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:59.407637 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69d3c85-ca68-4042-bc5f-ada7ea6fc9ac admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:59.493699 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-d69d3c85-ca68-4042-bc5f-ada7ea6fc9ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1cfed5b12c0402ba50f1a7ad230e891', 'request_id': 'req-d69d3c85-ca68-4042-bc5f-ada7ea6fc9ac'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:59.567917 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-d69d3c85-ca68-4042-bc5f-ada7ea6fc9ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1cfed5b12c0402ba50f1a7ad230e891', 'request_id': 'req-d69d3c85-ca68-4042-bc5f-ada7ea6fc9ac'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:59.573787 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-d69d3c85-ca68-4042-bc5f-ada7ea6fc9ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1cfed5b12c0402ba50f1a7ad230e891', 'request_id': 'req-d69d3c85-ca68-4042-bc5f-ada7ea6fc9ac'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:59.585712 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-d69d3c85-ca68-4042-bc5f-ada7ea6fc9ac admin admin] The token cache is being invalidate because project d1cfed5b12c0402ba50f1a7ad230e891 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:55:59.585993 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-d69d3c85-ca68-4042-bc5f-ada7ea6fc9ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:59.586710 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-d69d3c85-ca68-4042-bc5f-ada7ea6fc9ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:59.588298 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1253/2524] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:55:59 2022] DELETE /identity/v3/projects/d1cfed5b12c0402ba50f1a7ad230e891 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 02 19:55:59.592313 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e39aa2b-f966-4a9a-90a9-2fc4962fdc40 service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:55:59.602034 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e39aa2b-f966-4a9a-90a9-2fc4962fdc40 service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:55:59.620859 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:55:59.659236 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6171b304-41d9-4fd7-8afe-8e5a2f826578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:55:59.660143 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6171b304-41d9-4fd7-8afe-8e5a2f826578 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:55:59.660432 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6171b304-41d9-4fd7-8afe-8e5a2f826578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:55:59.660709 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6171b304-41d9-4fd7-8afe-8e5a2f826578 admin admin] PATH_INFO: `/v3/users/95396447fd51488d86be26fc447b99ac` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:55:59.661894 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6171b304-41d9-4fd7-8afe-8e5a2f826578 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95396447fd51488d86be26fc447b99ac)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:55:59.663877 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6171b304-41d9-4fd7-8afe-8e5a2f826578 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:55:59.831429 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-6171b304-41d9-4fd7-8afe-8e5a2f826578 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95396447fd51488d86be26fc447b99ac'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:59.839042 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-6171b304-41d9-4fd7-8afe-8e5a2f826578 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95396447fd51488d86be26fc447b99ac'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:59.898311 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-6171b304-41d9-4fd7-8afe-8e5a2f826578 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95396447fd51488d86be26fc447b99ac'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:59.899005 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-6171b304-41d9-4fd7-8afe-8e5a2f826578 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95396447fd51488d86be26fc447b99ac'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:59.900237 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-6171b304-41d9-4fd7-8afe-8e5a2f826578 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95396447fd51488d86be26fc447b99ac'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:55:59.918840 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1272/2525] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:55:59 2022] DELETE /identity/v3/users/95396447fd51488d86be26fc447b99ac => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:56:00.323298 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69d3c85-ca68-4042-bc5f-ada7ea6fc9ac admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:00.324885 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-d69d3c85-ca68-4042-bc5f-ada7ea6fc9ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:56:00.338947 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69d3c85-ca68-4042-bc5f-ada7ea6fc9ac admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:00.373970 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:56:00.400172 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ebc9b6-a2d4-4de7-9f09-41f069aa1777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:00.401032 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ebc9b6-a2d4-4de7-9f09-41f069aa1777 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:00.401326 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ebc9b6-a2d4-4de7-9f09-41f069aa1777 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:00.401608 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ebc9b6-a2d4-4de7-9f09-41f069aa1777 admin admin] PATH_INFO: `/v3/projects/8aa19f9ab3c94a30969cbec3d6ea805c` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:00.402818 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ebc9b6-a2d4-4de7-9f09-41f069aa1777 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8aa19f9ab3c94a30969cbec3d6ea805c)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:00.404727 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ebc9b6-a2d4-4de7-9f09-41f069aa1777 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:00.516364 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-43ebc9b6-a2d4-4de7-9f09-41f069aa1777 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8aa19f9ab3c94a30969cbec3d6ea805c', 'request_id': 'req-43ebc9b6-a2d4-4de7-9f09-41f069aa1777'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:00.600704 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-43ebc9b6-a2d4-4de7-9f09-41f069aa1777 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8aa19f9ab3c94a30969cbec3d6ea805c', 'request_id': 'req-43ebc9b6-a2d4-4de7-9f09-41f069aa1777'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:00.606608 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-43ebc9b6-a2d4-4de7-9f09-41f069aa1777 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8aa19f9ab3c94a30969cbec3d6ea805c', 'request_id': 'req-43ebc9b6-a2d4-4de7-9f09-41f069aa1777'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:00.617899 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-43ebc9b6-a2d4-4de7-9f09-41f069aa1777 admin admin] The token cache is being invalidate because project 8aa19f9ab3c94a30969cbec3d6ea805c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:56:00.618190 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-43ebc9b6-a2d4-4de7-9f09-41f069aa1777 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:00.618865 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-43ebc9b6-a2d4-4de7-9f09-41f069aa1777 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:00.620419 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1254/2526] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:56:00 2022] DELETE /identity/v3/projects/8aa19f9ab3c94a30969cbec3d6ea805c => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:56:00.628993 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce94fa-b3da-4057-9b6a-c461bdb81386 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:00.628993 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce94fa-b3da-4057-9b6a-c461bdb81386 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:00.628993 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce94fa-b3da-4057-9b6a-c461bdb81386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:00.655929 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-34ce94fa-b3da-4057-9b6a-c461bdb81386 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:56:00.655929 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-34ce94fa-b3da-4057-9b6a-c461bdb81386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:56:00.715472 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1273/2527] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:56:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:56:00.724679 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d8bbca-8239-442b-954c-cd9064f9b295 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:00.725017 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d8bbca-8239-442b-954c-cd9064f9b295 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:00.725247 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d8bbca-8239-442b-954c-cd9064f9b295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:00.744843 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-c4d8bbca-8239-442b-954c-cd9064f9b295 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:56:00.746056 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-c4d8bbca-8239-442b-954c-cd9064f9b295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:56:00.757312 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1255/2528] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:56:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 19:56:00.759314 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ce94fa-b3da-4057-9b6a-c461bdb81386 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:00.768986 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ce94fa-b3da-4057-9b6a-c461bdb81386 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:00.773609 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c7ca55-2024-4ef2-97d8-62bb714eceda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:00.774287 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c7ca55-2024-4ef2-97d8-62bb714eceda admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:00.774568 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c7ca55-2024-4ef2-97d8-62bb714eceda admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:00.774875 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c7ca55-2024-4ef2-97d8-62bb714eceda admin admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:00.775603 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c7ca55-2024-4ef2-97d8-62bb714eceda admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:56:00.775909 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c7ca55-2024-4ef2-97d8-62bb714eceda admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:00.777494 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c7ca55-2024-4ef2-97d8-62bb714eceda admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:00.788924 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1274/2529] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 2 19:56:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:56:00.793315 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d8bbca-8239-442b-954c-cd9064f9b295 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:00.802655 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d8bbca-8239-442b-954c-cd9064f9b295 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:00.807906 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1503bd03-8cef-423d-bd65-66820b01181e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:00.808551 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503bd03-8cef-423d-bd65-66820b01181e admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:00.808837 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503bd03-8cef-423d-bd65-66820b01181e admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:00.809117 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503bd03-8cef-423d-bd65-66820b01181e admin admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:00.809813 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1503bd03-8cef-423d-bd65-66820b01181e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:00.811778 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1503bd03-8cef-423d-bd65-66820b01181e admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:00.938941 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1256/2530] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:56:00 2022] POST /identity/v3/projects => generated 375 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:56:00.940152 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c7ca55-2024-4ef2-97d8-62bb714eceda admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:00.951122 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c7ca55-2024-4ef2-97d8-62bb714eceda admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:00.980531 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:56:01.010927 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57b351e-fd14-427c-8b41-ea20dcda452e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:01.011599 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b351e-fd14-427c-8b41-ea20dcda452e admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:01.011923 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b351e-fd14-427c-8b41-ea20dcda452e admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:01.012156 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b351e-fd14-427c-8b41-ea20dcda452e admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:01.012839 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57b351e-fd14-427c-8b41-ea20dcda452e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:01.015032 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57b351e-fd14-427c-8b41-ea20dcda452e admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:01.128914 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1275/2531] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:56:00 2022] POST /identity/v3/users => generated 288 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:56:01.132870 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1503bd03-8cef-423d-bd65-66820b01181e admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:01.143687 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1503bd03-8cef-423d-bd65-66820b01181e admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:01.148163 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea0e81b-6201-47f9-be54-b52904ab31dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:01.148813 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea0e81b-6201-47f9-be54-b52904ab31dc admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:01.149758 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea0e81b-6201-47f9-be54-b52904ab31dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:01.150041 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea0e81b-6201-47f9-be54-b52904ab31dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:01.150780 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea0e81b-6201-47f9-be54-b52904ab31dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:01.152389 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea0e81b-6201-47f9-be54-b52904ab31dc admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:01.162492 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1257/2532] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:56:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 02 19:56:01.166781 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57b351e-fd14-427c-8b41-ea20dcda452e admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:01.176767 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57b351e-fd14-427c-8b41-ea20dcda452e admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:01.181570 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c17b1d9-0f1c-4eed-bffe-a5a008760479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:01.182214 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c17b1d9-0f1c-4eed-bffe-a5a008760479 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:01.182530 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c17b1d9-0f1c-4eed-bffe-a5a008760479 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:01.182848 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c17b1d9-0f1c-4eed-bffe-a5a008760479 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:01.183582 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c17b1d9-0f1c-4eed-bffe-a5a008760479 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:01.185189 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c17b1d9-0f1c-4eed-bffe-a5a008760479 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:01.196861 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1276/2533] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:56:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:56:01.200554 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea0e81b-6201-47f9-be54-b52904ab31dc admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:01.210415 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea0e81b-6201-47f9-be54-b52904ab31dc admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:01.215652 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60146884-fe72-4756-a4b2-1f9de2c9506f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:01.216539 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60146884-fe72-4756-a4b2-1f9de2c9506f admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:01.216825 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60146884-fe72-4756-a4b2-1f9de2c9506f admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:01.217106 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60146884-fe72-4756-a4b2-1f9de2c9506f admin admin] PATH_INFO: `/v3/projects/7d7a3226e4b8464ea86c242e0d83d975/users/77c72931371e4f00a66e848295d70351/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:01.232823 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60146884-fe72-4756-a4b2-1f9de2c9506f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d7a3226e4b8464ea86c242e0d83d975, user_id=77c72931371e4f00a66e848295d70351, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:01.235542 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60146884-fe72-4756-a4b2-1f9de2c9506f admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:01.316221 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1258/2534] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:56:01 2022] PUT /identity/v3/projects/7d7a3226e4b8464ea86c242e0d83d975/users/77c72931371e4f00a66e848295d70351/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:56:01.327635 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4863592-db82-4f6d-90a5-a86d6f93a4d2 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:01.327925 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4863592-db82-4f6d-90a5-a86d6f93a4d2 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:01.328203 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4863592-db82-4f6d-90a5-a86d6f93a4d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:01.354305 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-b4863592-db82-4f6d-90a5-a86d6f93a4d2 None None] MFA Rules not processed for user `77c72931371e4f00a66e848295d70351`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:56:01.355536 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-b4863592-db82-4f6d-90a5-a86d6f93a4d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:56:01.420858 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1277/2535] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:56:01 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:56:01.432227 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60146884-fe72-4756-a4b2-1f9de2c9506f admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:01.434053 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-60146884-fe72-4756-a4b2-1f9de2c9506f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:56:01.490081 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60146884-fe72-4756-a4b2-1f9de2c9506f admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:01.493752 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20336fbc-a36b-4035-b6fa-6a28a4fc3613 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:01.494392 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20336fbc-a36b-4035-b6fa-6a28a4fc3613 service cinder] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:01.494681 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20336fbc-a36b-4035-b6fa-6a28a4fc3613 service cinder] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:01.494993 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20336fbc-a36b-4035-b6fa-6a28a4fc3613 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:01.503192 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20336fbc-a36b-4035-b6fa-6a28a4fc3613 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:01.504621 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20336fbc-a36b-4035-b6fa-6a28a4fc3613 service cinder] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:01.513333 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1259/2536] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:56:01 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:56:02.018561 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4863592-db82-4f6d-90a5-a86d6f93a4d2 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:02.020306 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-b4863592-db82-4f6d-90a5-a86d6f93a4d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:56:02.033159 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4863592-db82-4f6d-90a5-a86d6f93a4d2 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:02.036744 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d239000-4fee-4f99-b7c0-28587314ce36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:02.037388 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d239000-4fee-4f99-b7c0-28587314ce36 service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:02.037676 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d239000-4fee-4f99-b7c0-28587314ce36 service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:02.037985 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d239000-4fee-4f99-b7c0-28587314ce36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:02.039778 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-1d239000-4fee-4f99-b7c0-28587314ce36 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:56:02.098167 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d239000-4fee-4f99-b7c0-28587314ce36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:02.099650 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d239000-4fee-4f99-b7c0-28587314ce36 service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:02.111210 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1278/2537] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:56:02 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:56:05.624214 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20336fbc-a36b-4035-b6fa-6a28a4fc3613 service cinder] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:05.625649 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-20336fbc-a36b-4035-b6fa-6a28a4fc3613 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:56:05.631136 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:56:05.693477 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20336fbc-a36b-4035-b6fa-6a28a4fc3613 service cinder] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:05.698774 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9045c2f0-c985-4ec6-abb0-e4bfd0c6810a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:05.699666 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9045c2f0-c985-4ec6-abb0-e4bfd0c6810a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:05.701338 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9045c2f0-c985-4ec6-abb0-e4bfd0c6810a admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:05.701338 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9045c2f0-c985-4ec6-abb0-e4bfd0c6810a admin admin] PATH_INFO: `/v3/users/5575aa83e93f491c88a7d1cf83e8d7d8` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:05.701556 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9045c2f0-c985-4ec6-abb0-e4bfd0c6810a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5575aa83e93f491c88a7d1cf83e8d7d8)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:05.703600 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9045c2f0-c985-4ec6-abb0-e4bfd0c6810a admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:05.910194 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-9045c2f0-c985-4ec6-abb0-e4bfd0c6810a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5575aa83e93f491c88a7d1cf83e8d7d8'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:05.916969 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-9045c2f0-c985-4ec6-abb0-e4bfd0c6810a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5575aa83e93f491c88a7d1cf83e8d7d8'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:05.917635 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-9045c2f0-c985-4ec6-abb0-e4bfd0c6810a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5575aa83e93f491c88a7d1cf83e8d7d8'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:05.977765 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-9045c2f0-c985-4ec6-abb0-e4bfd0c6810a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5575aa83e93f491c88a7d1cf83e8d7d8'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:05.978385 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-9045c2f0-c985-4ec6-abb0-e4bfd0c6810a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5575aa83e93f491c88a7d1cf83e8d7d8'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:05.996039 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1260/2538] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:56:05 2022] DELETE /identity/v3/users/5575aa83e93f491c88a7d1cf83e8d7d8 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:56:06.000511 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874bdfa1-f5fa-4287-a556-17bddb0fdca5 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:06.000958 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874bdfa1-f5fa-4287-a556-17bddb0fdca5 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:06.001066 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874bdfa1-f5fa-4287-a556-17bddb0fdca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:06.018765 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-874bdfa1-f5fa-4287-a556-17bddb0fdca5 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:56:06.019976 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-874bdfa1-f5fa-4287-a556-17bddb0fdca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:56:06.049348 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:56:06.078384 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1279/2539] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:56:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:56:06.159392 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9045c2f0-c985-4ec6-abb0-e4bfd0c6810a admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:06.160847 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-9045c2f0-c985-4ec6-abb0-e4bfd0c6810a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:56:06.348925 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9045c2f0-c985-4ec6-abb0-e4bfd0c6810a admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:06.348925 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a8274f-5f62-42bf-9e8f-86fe23dae616 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:06.348925 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a8274f-5f62-42bf-9e8f-86fe23dae616 service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:06.348925 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a8274f-5f62-42bf-9e8f-86fe23dae616 service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:06.349688 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a8274f-5f62-42bf-9e8f-86fe23dae616 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:06.349688 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a8274f-5f62-42bf-9e8f-86fe23dae616 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:06.349688 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a8274f-5f62-42bf-9e8f-86fe23dae616 service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:06.349688 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1261/2540] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:56:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:56:06.829883 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874bdfa1-f5fa-4287-a556-17bddb0fdca5 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:06.831343 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-874bdfa1-f5fa-4287-a556-17bddb0fdca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:56:06.845223 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874bdfa1-f5fa-4287-a556-17bddb0fdca5 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:06.850230 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba54093-bdc8-48ed-a373-c81f95d8fe0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:06.851195 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba54093-bdc8-48ed-a373-c81f95d8fe0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:06.851506 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba54093-bdc8-48ed-a373-c81f95d8fe0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:06.851806 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba54093-bdc8-48ed-a373-c81f95d8fe0c admin admin] PATH_INFO: `/v3/projects/a6dfa1fcf1ad4fde94627804a8b87ea7` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:06.853003 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba54093-bdc8-48ed-a373-c81f95d8fe0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6dfa1fcf1ad4fde94627804a8b87ea7)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:06.855026 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba54093-bdc8-48ed-a373-c81f95d8fe0c admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:06.999840 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-1ba54093-bdc8-48ed-a373-c81f95d8fe0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6dfa1fcf1ad4fde94627804a8b87ea7', 'request_id': 'req-1ba54093-bdc8-48ed-a373-c81f95d8fe0c'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:07.079259 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-1ba54093-bdc8-48ed-a373-c81f95d8fe0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6dfa1fcf1ad4fde94627804a8b87ea7', 'request_id': 'req-1ba54093-bdc8-48ed-a373-c81f95d8fe0c'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:07.084965 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-1ba54093-bdc8-48ed-a373-c81f95d8fe0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6dfa1fcf1ad4fde94627804a8b87ea7', 'request_id': 'req-1ba54093-bdc8-48ed-a373-c81f95d8fe0c'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:07.096001 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-1ba54093-bdc8-48ed-a373-c81f95d8fe0c admin admin] The token cache is being invalidate because project a6dfa1fcf1ad4fde94627804a8b87ea7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:56:07.100898 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-1ba54093-bdc8-48ed-a373-c81f95d8fe0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:07.100898 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-1ba54093-bdc8-48ed-a373-c81f95d8fe0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:07.100898 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1280/2541] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:56:06 2022] DELETE /identity/v3/projects/a6dfa1fcf1ad4fde94627804a8b87ea7 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:56:07.103044 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a8274f-5f62-42bf-9e8f-86fe23dae616 service neutron] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:07.112852 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a8274f-5f62-42bf-9e8f-86fe23dae616 service neutron] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:07.134296 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:56:07.169777 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0592e2f6-549f-40a8-93db-235e4132fc7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:07.170655 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0592e2f6-549f-40a8-93db-235e4132fc7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:07.171172 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0592e2f6-549f-40a8-93db-235e4132fc7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:07.171515 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0592e2f6-549f-40a8-93db-235e4132fc7b admin admin] PATH_INFO: `/v3/users/cdfe8b0a0cee46b186022b0384ea66ea` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:07.173107 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0592e2f6-549f-40a8-93db-235e4132fc7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdfe8b0a0cee46b186022b0384ea66ea)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:07.175184 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0592e2f6-549f-40a8-93db-235e4132fc7b admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:07.352213 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-0592e2f6-549f-40a8-93db-235e4132fc7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdfe8b0a0cee46b186022b0384ea66ea'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:07.359423 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-0592e2f6-549f-40a8-93db-235e4132fc7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdfe8b0a0cee46b186022b0384ea66ea'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:07.360107 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-0592e2f6-549f-40a8-93db-235e4132fc7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdfe8b0a0cee46b186022b0384ea66ea'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:07.428099 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-0592e2f6-549f-40a8-93db-235e4132fc7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdfe8b0a0cee46b186022b0384ea66ea'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:07.428761 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-0592e2f6-549f-40a8-93db-235e4132fc7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdfe8b0a0cee46b186022b0384ea66ea'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:07.447719 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1262/2542] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:56:07 2022] DELETE /identity/v3/users/cdfe8b0a0cee46b186022b0384ea66ea => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:56:07.865389 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba54093-bdc8-48ed-a373-c81f95d8fe0c admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:07.866865 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-1ba54093-bdc8-48ed-a373-c81f95d8fe0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:56:07.880913 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba54093-bdc8-48ed-a373-c81f95d8fe0c admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:07.940921 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550a1151-2042-4684-bf38-8a798ae96d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:07.941793 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550a1151-2042-4684-bf38-8a798ae96d59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:07.942137 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550a1151-2042-4684-bf38-8a798ae96d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:07.942425 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550a1151-2042-4684-bf38-8a798ae96d59 admin admin] PATH_INFO: `/v3/projects/5d12fe55754f46ff84edef8863bea044` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:07.943653 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550a1151-2042-4684-bf38-8a798ae96d59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d12fe55754f46ff84edef8863bea044)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:07.945608 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550a1151-2042-4684-bf38-8a798ae96d59 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:08.057299 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-550a1151-2042-4684-bf38-8a798ae96d59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d12fe55754f46ff84edef8863bea044', 'request_id': 'req-550a1151-2042-4684-bf38-8a798ae96d59'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:08.148306 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-550a1151-2042-4684-bf38-8a798ae96d59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d12fe55754f46ff84edef8863bea044', 'request_id': 'req-550a1151-2042-4684-bf38-8a798ae96d59'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:08.153261 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:56:08.155487 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-550a1151-2042-4684-bf38-8a798ae96d59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d12fe55754f46ff84edef8863bea044', 'request_id': 'req-550a1151-2042-4684-bf38-8a798ae96d59'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:08.167123 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-550a1151-2042-4684-bf38-8a798ae96d59 admin admin] The token cache is being invalidate because project 5d12fe55754f46ff84edef8863bea044 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:56:08.167413 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-550a1151-2042-4684-bf38-8a798ae96d59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:08.168074 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-550a1151-2042-4684-bf38-8a798ae96d59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:08.169620 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1281/2543] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:56:07 2022] DELETE /identity/v3/projects/5d12fe55754f46ff84edef8863bea044 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:56:08.173700 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0592e2f6-549f-40a8-93db-235e4132fc7b admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:08.175452 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-0592e2f6-549f-40a8-93db-235e4132fc7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:56:08.187690 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0592e2f6-549f-40a8-93db-235e4132fc7b admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:08.244516 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d99e68-a814-4d8e-bae4-8befb6978bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:08.245435 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d99e68-a814-4d8e-bae4-8befb6978bbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:08.245720 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d99e68-a814-4d8e-bae4-8befb6978bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:08.245998 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d99e68-a814-4d8e-bae4-8befb6978bbd admin admin] PATH_INFO: `/v3/users/a8a20bcf1ca34a9093608ab426305dbf` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:08.247259 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d99e68-a814-4d8e-bae4-8befb6978bbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8a20bcf1ca34a9093608ab426305dbf)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:08.249311 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d99e68-a814-4d8e-bae4-8befb6978bbd admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:08.475808 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:56:08.478090 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-27d99e68-a814-4d8e-bae4-8befb6978bbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8a20bcf1ca34a9093608ab426305dbf'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:08.484810 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-27d99e68-a814-4d8e-bae4-8befb6978bbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8a20bcf1ca34a9093608ab426305dbf'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:08.485481 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-27d99e68-a814-4d8e-bae4-8befb6978bbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8a20bcf1ca34a9093608ab426305dbf'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:08.577342 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-27d99e68-a814-4d8e-bae4-8befb6978bbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8a20bcf1ca34a9093608ab426305dbf'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:08.577979 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-27d99e68-a814-4d8e-bae4-8befb6978bbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8a20bcf1ca34a9093608ab426305dbf'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:08.596038 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1263/2544] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:56:08 2022] DELETE /identity/v3/users/a8a20bcf1ca34a9093608ab426305dbf => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 02 19:56:09.024310 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550a1151-2042-4684-bf38-8a798ae96d59 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:09.025757 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-550a1151-2042-4684-bf38-8a798ae96d59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:56:09.038655 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550a1151-2042-4684-bf38-8a798ae96d59 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:09.095614 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f981b263-447f-46a1-bd9a-011ce71a7868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:09.096479 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f981b263-447f-46a1-bd9a-011ce71a7868 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:09.096764 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f981b263-447f-46a1-bd9a-011ce71a7868 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:09.097043 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f981b263-447f-46a1-bd9a-011ce71a7868 admin admin] PATH_INFO: `/v3/projects/e91a30768c2440488f501283f11121ed` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:09.098206 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f981b263-447f-46a1-bd9a-011ce71a7868 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e91a30768c2440488f501283f11121ed)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:09.100175 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f981b263-447f-46a1-bd9a-011ce71a7868 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:09.219334 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f981b263-447f-46a1-bd9a-011ce71a7868 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e91a30768c2440488f501283f11121ed', 'request_id': 'req-f981b263-447f-46a1-bd9a-011ce71a7868'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:09.269353 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f981b263-447f-46a1-bd9a-011ce71a7868 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e91a30768c2440488f501283f11121ed', 'request_id': 'req-f981b263-447f-46a1-bd9a-011ce71a7868'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:09.276285 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f981b263-447f-46a1-bd9a-011ce71a7868 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e91a30768c2440488f501283f11121ed', 'request_id': 'req-f981b263-447f-46a1-bd9a-011ce71a7868'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:09.288178 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f981b263-447f-46a1-bd9a-011ce71a7868 admin admin] The token cache is being invalidate because project e91a30768c2440488f501283f11121ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:56:09.288476 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f981b263-447f-46a1-bd9a-011ce71a7868 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:09.289716 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f981b263-447f-46a1-bd9a-011ce71a7868 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:09.291629 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1282/2545] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:56:09 2022] DELETE /identity/v3/projects/e91a30768c2440488f501283f11121ed => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:56:09.297534 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065c361-5919-4672-8122-92a8fb360872 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:09.298005 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065c361-5919-4672-8122-92a8fb360872 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:09.298079 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065c361-5919-4672-8122-92a8fb360872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:09.318237 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-9065c361-5919-4672-8122-92a8fb360872 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:56:09.319578 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-9065c361-5919-4672-8122-92a8fb360872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:56:09.378901 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1264/2546] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:56:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:56:09.387774 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da620364-0f6c-4a91-b2eb-b69617497466 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:09.388072 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da620364-0f6c-4a91-b2eb-b69617497466 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:09.388356 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da620364-0f6c-4a91-b2eb-b69617497466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:09.406153 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-da620364-0f6c-4a91-b2eb-b69617497466 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:56:09.407413 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-da620364-0f6c-4a91-b2eb-b69617497466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:56:09.415760 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1283/2547] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:56:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:56:09.419751 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9065c361-5919-4672-8122-92a8fb360872 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:09.429431 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9065c361-5919-4672-8122-92a8fb360872 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:09.434555 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6535ee-59b8-47d9-b3db-7062691aa698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:09.435236 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6535ee-59b8-47d9-b3db-7062691aa698 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:09.435520 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6535ee-59b8-47d9-b3db-7062691aa698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:09.435798 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6535ee-59b8-47d9-b3db-7062691aa698 admin admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:09.436521 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6535ee-59b8-47d9-b3db-7062691aa698 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:56:09.436830 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6535ee-59b8-47d9-b3db-7062691aa698 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:09.438411 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6535ee-59b8-47d9-b3db-7062691aa698 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:09.453159 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1265/2548] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 2 19:56:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:56:09.454450 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da620364-0f6c-4a91-b2eb-b69617497466 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:09.463795 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da620364-0f6c-4a91-b2eb-b69617497466 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:09.468651 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5615fec8-ab24-430c-99a5-325e29bd4360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:09.469307 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615fec8-ab24-430c-99a5-325e29bd4360 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:09.469647 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615fec8-ab24-430c-99a5-325e29bd4360 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:09.469875 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615fec8-ab24-430c-99a5-325e29bd4360 admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:09.470605 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5615fec8-ab24-430c-99a5-325e29bd4360 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:09.472555 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5615fec8-ab24-430c-99a5-325e29bd4360 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:09.544565 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1284/2549] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:56:09 2022] POST /identity/v3/projects => generated 393 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:56:09.548616 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6535ee-59b8-47d9-b3db-7062691aa698 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:09.559220 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6535ee-59b8-47d9-b3db-7062691aa698 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:09.615386 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f1e272-f16b-430a-ba19-7d7a416af4cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:09.616067 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f1e272-f16b-430a-ba19-7d7a416af4cc admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:09.616355 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f1e272-f16b-430a-ba19-7d7a416af4cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:09.616636 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f1e272-f16b-430a-ba19-7d7a416af4cc admin admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:09.617973 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f1e272-f16b-430a-ba19-7d7a416af4cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:09.619965 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f1e272-f16b-430a-ba19-7d7a416af4cc admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:09.701871 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1266/2550] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:56:09 2022] POST /identity/v3/users => generated 297 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:56:09.705961 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5615fec8-ab24-430c-99a5-325e29bd4360 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:09.716592 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5615fec8-ab24-430c-99a5-325e29bd4360 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:09.721217 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe23e8c4-d0ba-4706-8360-d000411ec6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:09.721810 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe23e8c4-d0ba-4706-8360-d000411ec6bd admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:09.722091 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe23e8c4-d0ba-4706-8360-d000411ec6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:09.722363 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe23e8c4-d0ba-4706-8360-d000411ec6bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:09.723121 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe23e8c4-d0ba-4706-8360-d000411ec6bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:09.728754 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe23e8c4-d0ba-4706-8360-d000411ec6bd admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:09.735406 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1285/2551] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:56:09.739211 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f1e272-f16b-430a-ba19-7d7a416af4cc admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:09.749139 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f1e272-f16b-430a-ba19-7d7a416af4cc admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:09.753840 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ec8df-3a9d-4de1-a8bc-eae9d4652dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:09.754497 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ec8df-3a9d-4de1-a8bc-eae9d4652dd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:09.754820 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ec8df-3a9d-4de1-a8bc-eae9d4652dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:09.755106 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ec8df-3a9d-4de1-a8bc-eae9d4652dd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:09.755838 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232ec8df-3a9d-4de1-a8bc-eae9d4652dd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:09.757428 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232ec8df-3a9d-4de1-a8bc-eae9d4652dd5 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:09.767455 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1267/2552] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:56:09.771348 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe23e8c4-d0ba-4706-8360-d000411ec6bd admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:09.781382 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe23e8c4-d0ba-4706-8360-d000411ec6bd admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:09.785867 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680662f4-7e1a-4c0b-a740-3a21250e9c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:09.786729 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680662f4-7e1a-4c0b-a740-3a21250e9c1e admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:09.787040 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680662f4-7e1a-4c0b-a740-3a21250e9c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:09.787313 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680662f4-7e1a-4c0b-a740-3a21250e9c1e admin admin] PATH_INFO: `/v3/projects/2a4dfbfa5e8b401b9e660269074f1002/users/b3c46f515cac464789b6ff442c4857ee/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:09.801903 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680662f4-7e1a-4c0b-a740-3a21250e9c1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a4dfbfa5e8b401b9e660269074f1002, user_id=b3c46f515cac464789b6ff442c4857ee, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:09.804548 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680662f4-7e1a-4c0b-a740-3a21250e9c1e admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:09.885352 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1286/2553] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:56:09 2022] PUT /identity/v3/projects/2a4dfbfa5e8b401b9e660269074f1002/users/b3c46f515cac464789b6ff442c4857ee/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:56:09.896741 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64437d6-295f-43de-802f-276db1cad52d None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:09.897032 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64437d6-295f-43de-802f-276db1cad52d None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:09.897315 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64437d6-295f-43de-802f-276db1cad52d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:09.923708 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-a64437d6-295f-43de-802f-276db1cad52d None None] MFA Rules not processed for user `b3c46f515cac464789b6ff442c4857ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:56:09.924906 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-a64437d6-295f-43de-802f-276db1cad52d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:56:09.941461 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:56:09.988475 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1268/2554] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:56:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:56:09.993387 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680662f4-7e1a-4c0b-a740-3a21250e9c1e admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:10.011743 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:56:10.056284 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680662f4-7e1a-4c0b-a740-3a21250e9c1e admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:10.060935 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d217293-7e43-47a6-b125-40d28cf2253c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:10.061594 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d217293-7e43-47a6-b125-40d28cf2253c admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:10.061879 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d217293-7e43-47a6-b125-40d28cf2253c admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:10.062156 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d217293-7e43-47a6-b125-40d28cf2253c admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:10.062874 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d217293-7e43-47a6-b125-40d28cf2253c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:10.064775 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d217293-7e43-47a6-b125-40d28cf2253c admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:10.168527 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1287/2555] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:56:09 2022] POST /identity/v3/projects => generated 393 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:56:10.177948 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64437d6-295f-43de-802f-276db1cad52d None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:10.189509 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64437d6-295f-43de-802f-276db1cad52d None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:10.245635 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5494aaf1-30ef-42a1-869f-46540fef99f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:10.246238 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5494aaf1-30ef-42a1-869f-46540fef99f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:10.246526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5494aaf1-30ef-42a1-869f-46540fef99f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:10.246837 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5494aaf1-30ef-42a1-869f-46540fef99f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:10.247611 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5494aaf1-30ef-42a1-869f-46540fef99f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:10.249530 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5494aaf1-30ef-42a1-869f-46540fef99f4 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:10.365527 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1269/2556] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:56:10 2022] POST /identity/v3/users => generated 303 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:56:10.370495 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d217293-7e43-47a6-b125-40d28cf2253c admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:10.380747 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d217293-7e43-47a6-b125-40d28cf2253c admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:10.385195 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749a0d76-ef18-43e9-b16a-230b2fd8e18c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:10.385852 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a0d76-ef18-43e9-b16a-230b2fd8e18c admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:10.386138 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a0d76-ef18-43e9-b16a-230b2fd8e18c admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:10.386447 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a0d76-ef18-43e9-b16a-230b2fd8e18c admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:10.387471 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749a0d76-ef18-43e9-b16a-230b2fd8e18c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:10.389072 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749a0d76-ef18-43e9-b16a-230b2fd8e18c admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:10.399956 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1288/2557] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:56:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 02 19:56:10.403918 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5494aaf1-30ef-42a1-869f-46540fef99f4 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:10.413662 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5494aaf1-30ef-42a1-869f-46540fef99f4 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:10.418881 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab77cb51-644b-4577-9d9a-e61b89e23797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:10.419784 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab77cb51-644b-4577-9d9a-e61b89e23797 admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:10.420070 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab77cb51-644b-4577-9d9a-e61b89e23797 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:10.420345 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab77cb51-644b-4577-9d9a-e61b89e23797 admin admin] PATH_INFO: `/v3/projects/3bce0ef58c2f4d3cad6fcd3e2d8d9b62/users/080a6a9092fb467eab61323a106a6e01/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:10.435914 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab77cb51-644b-4577-9d9a-e61b89e23797 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bce0ef58c2f4d3cad6fcd3e2d8d9b62, user_id=080a6a9092fb467eab61323a106a6e01, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:10.438559 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab77cb51-644b-4577-9d9a-e61b89e23797 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:10.510211 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1270/2558] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:56:10 2022] PUT /identity/v3/projects/3bce0ef58c2f4d3cad6fcd3e2d8d9b62/users/080a6a9092fb467eab61323a106a6e01/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:56:10.514534 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749a0d76-ef18-43e9-b16a-230b2fd8e18c admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:10.525514 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749a0d76-ef18-43e9-b16a-230b2fd8e18c admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:10.583103 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1893290-15c7-4d42-8149-5617ee32f7a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:10.583772 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1893290-15c7-4d42-8149-5617ee32f7a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:10.584058 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1893290-15c7-4d42-8149-5617ee32f7a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:10.584335 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1893290-15c7-4d42-8149-5617ee32f7a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:10.585063 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1893290-15c7-4d42-8149-5617ee32f7a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:10.587202 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1893290-15c7-4d42-8149-5617ee32f7a1 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:10.596700 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1289/2559] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:56:10 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:56:10.600828 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab77cb51-644b-4577-9d9a-e61b89e23797 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:10.611660 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab77cb51-644b-4577-9d9a-e61b89e23797 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:10.616167 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8336fbb-4e6f-4c9f-996c-c7cf3a44b1a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:10.617034 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8336fbb-4e6f-4c9f-996c-c7cf3a44b1a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:10.617319 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8336fbb-4e6f-4c9f-996c-c7cf3a44b1a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:10.617596 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8336fbb-4e6f-4c9f-996c-c7cf3a44b1a3 admin admin] PATH_INFO: `/v3/projects/3bce0ef58c2f4d3cad6fcd3e2d8d9b62/users/080a6a9092fb467eab61323a106a6e01/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:10.619559 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8336fbb-4e6f-4c9f-996c-c7cf3a44b1a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bce0ef58c2f4d3cad6fcd3e2d8d9b62, user_id=080a6a9092fb467eab61323a106a6e01, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:10.622136 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8336fbb-4e6f-4c9f-996c-c7cf3a44b1a3 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:10.632121 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1271/2560] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:56:10 2022] PUT /identity/v3/projects/3bce0ef58c2f4d3cad6fcd3e2d8d9b62/users/080a6a9092fb467eab61323a106a6e01/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:56:10.643083 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eaa3bc-213a-48dd-994a-cf1b6a261b72 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:10.643379 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eaa3bc-213a-48dd-994a-cf1b6a261b72 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:10.643661 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eaa3bc-213a-48dd-994a-cf1b6a261b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:10.669982 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-c3eaa3bc-213a-48dd-994a-cf1b6a261b72 None None] MFA Rules not processed for user `080a6a9092fb467eab61323a106a6e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:56:10.671224 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-c3eaa3bc-213a-48dd-994a-cf1b6a261b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:56:10.737302 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1290/2561] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:56:10 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:56:10.748220 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8336fbb-4e6f-4c9f-996c-c7cf3a44b1a3 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:10.749849 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-a8336fbb-4e6f-4c9f-996c-c7cf3a44b1a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:56:10.805954 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8336fbb-4e6f-4c9f-996c-c7cf3a44b1a3 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:10.809508 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f052f25-4d14-47e3-aa5b-062202a2da9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:10.810139 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f052f25-4d14-47e3-aa5b-062202a2da9f service cinder] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:10.810421 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f052f25-4d14-47e3-aa5b-062202a2da9f service cinder] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:10.810694 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f052f25-4d14-47e3-aa5b-062202a2da9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:10.819644 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f052f25-4d14-47e3-aa5b-062202a2da9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:10.821560 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f052f25-4d14-47e3-aa5b-062202a2da9f service cinder] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:10.830726 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1272/2562] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:56:10 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:56:10.874642 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eaa3bc-213a-48dd-994a-cf1b6a261b72 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:10.876056 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-c3eaa3bc-213a-48dd-994a-cf1b6a261b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:56:10.888604 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eaa3bc-213a-48dd-994a-cf1b6a261b72 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:10.892235 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f296df0-2b62-4b09-a95a-9a2c4173e007 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:10.892870 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f296df0-2b62-4b09-a95a-9a2c4173e007 service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:10.893162 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f296df0-2b62-4b09-a95a-9a2c4173e007 service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:10.893440 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f296df0-2b62-4b09-a95a-9a2c4173e007 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:10.902258 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:56:10.950504 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f296df0-2b62-4b09-a95a-9a2c4173e007 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:10.951998 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f296df0-2b62-4b09-a95a-9a2c4173e007 service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:10.961416 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1291/2563] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:56:10 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:56:15.727576 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f052f25-4d14-47e3-aa5b-062202a2da9f service cinder] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:15.740443 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:56:15.789710 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f052f25-4d14-47e3-aa5b-062202a2da9f service cinder] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:15.794527 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3746bd-fd36-4be7-a292-ba8a34db1f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:15.796268 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3746bd-fd36-4be7-a292-ba8a34db1f24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:15.796268 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3746bd-fd36-4be7-a292-ba8a34db1f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:15.796268 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3746bd-fd36-4be7-a292-ba8a34db1f24 admin admin] PATH_INFO: `/v3/users/b3c46f515cac464789b6ff442c4857ee` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:15.797330 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3746bd-fd36-4be7-a292-ba8a34db1f24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3c46f515cac464789b6ff442c4857ee)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:15.799276 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3746bd-fd36-4be7-a292-ba8a34db1f24 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:16.000168 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-3e3746bd-fd36-4be7-a292-ba8a34db1f24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3c46f515cac464789b6ff442c4857ee'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:16.007141 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-3e3746bd-fd36-4be7-a292-ba8a34db1f24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3c46f515cac464789b6ff442c4857ee'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:16.008315 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-3e3746bd-fd36-4be7-a292-ba8a34db1f24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3c46f515cac464789b6ff442c4857ee'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:16.067192 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-3e3746bd-fd36-4be7-a292-ba8a34db1f24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3c46f515cac464789b6ff442c4857ee'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:16.067816 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-3e3746bd-fd36-4be7-a292-ba8a34db1f24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3c46f515cac464789b6ff442c4857ee'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:16.086262 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1273/2564] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:56:15 2022] DELETE /identity/v3/users/b3c46f515cac464789b6ff442c4857ee => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:56:16.090830 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6148bf-b94c-4d9e-a8fe-bdc4c1d080af None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:16.091310 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6148bf-b94c-4d9e-a8fe-bdc4c1d080af None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:16.091419 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6148bf-b94c-4d9e-a8fe-bdc4c1d080af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:16.108156 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-6e6148bf-b94c-4d9e-a8fe-bdc4c1d080af None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:56:16.109365 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-6e6148bf-b94c-4d9e-a8fe-bdc4c1d080af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:56:16.168298 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1292/2565] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:56:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:56:16.177852 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3746bd-fd36-4be7-a292-ba8a34db1f24 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:16.179266 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-3e3746bd-fd36-4be7-a292-ba8a34db1f24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:56:16.235159 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3746bd-fd36-4be7-a292-ba8a34db1f24 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:16.238712 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e689a4-9f51-4292-913f-f46dfb3d5bca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:16.239386 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e689a4-9f51-4292-913f-f46dfb3d5bca service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:16.239669 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e689a4-9f51-4292-913f-f46dfb3d5bca service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:16.239953 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e689a4-9f51-4292-913f-f46dfb3d5bca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:16.249009 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e689a4-9f51-4292-913f-f46dfb3d5bca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:16.250504 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e689a4-9f51-4292-913f-f46dfb3d5bca service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:16.260510 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1274/2566] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:56:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:56:16.667861 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6148bf-b94c-4d9e-a8fe-bdc4c1d080af None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:16.670225 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-6e6148bf-b94c-4d9e-a8fe-bdc4c1d080af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:56:16.683295 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6148bf-b94c-4d9e-a8fe-bdc4c1d080af None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:16.688420 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361fcdca-20e7-4b04-ac52-6a41a5692584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:16.689047 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361fcdca-20e7-4b04-ac52-6a41a5692584 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:16.689268 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361fcdca-20e7-4b04-ac52-6a41a5692584 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:16.689552 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361fcdca-20e7-4b04-ac52-6a41a5692584 admin admin] PATH_INFO: `/v3/projects/2a4dfbfa5e8b401b9e660269074f1002` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:16.690821 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361fcdca-20e7-4b04-ac52-6a41a5692584 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a4dfbfa5e8b401b9e660269074f1002)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:16.692902 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361fcdca-20e7-4b04-ac52-6a41a5692584 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:16.810787 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-361fcdca-20e7-4b04-ac52-6a41a5692584 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a4dfbfa5e8b401b9e660269074f1002', 'request_id': 'req-361fcdca-20e7-4b04-ac52-6a41a5692584'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:16.873258 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-361fcdca-20e7-4b04-ac52-6a41a5692584 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a4dfbfa5e8b401b9e660269074f1002', 'request_id': 'req-361fcdca-20e7-4b04-ac52-6a41a5692584'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:16.879546 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-361fcdca-20e7-4b04-ac52-6a41a5692584 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a4dfbfa5e8b401b9e660269074f1002', 'request_id': 'req-361fcdca-20e7-4b04-ac52-6a41a5692584'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:16.891319 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-361fcdca-20e7-4b04-ac52-6a41a5692584 admin admin] The token cache is being invalidate because project 2a4dfbfa5e8b401b9e660269074f1002 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:56:16.891612 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-361fcdca-20e7-4b04-ac52-6a41a5692584 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:16.892247 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-361fcdca-20e7-4b04-ac52-6a41a5692584 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:16.894025 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1293/2567] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:56:16 2022] DELETE /identity/v3/projects/2a4dfbfa5e8b401b9e660269074f1002 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:56:16.897967 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e689a4-9f51-4292-913f-f46dfb3d5bca service neutron] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:16.907523 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e689a4-9f51-4292-913f-f46dfb3d5bca service neutron] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:16.940762 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:56:16.963870 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614c7b94-362b-4171-90f3-b511c370880b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:16.964728 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614c7b94-362b-4171-90f3-b511c370880b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:16.965007 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614c7b94-362b-4171-90f3-b511c370880b admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:16.965279 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614c7b94-362b-4171-90f3-b511c370880b admin admin] PATH_INFO: `/v3/users/080a6a9092fb467eab61323a106a6e01` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:16.966441 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614c7b94-362b-4171-90f3-b511c370880b admin admin] RBAC: Authorizing `identity:delete_user(user_id=080a6a9092fb467eab61323a106a6e01)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:16.968378 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614c7b94-362b-4171-90f3-b511c370880b admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:17.138289 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-614c7b94-362b-4171-90f3-b511c370880b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '080a6a9092fb467eab61323a106a6e01'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:17.145315 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-614c7b94-362b-4171-90f3-b511c370880b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '080a6a9092fb467eab61323a106a6e01'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:17.146029 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-614c7b94-362b-4171-90f3-b511c370880b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '080a6a9092fb467eab61323a106a6e01'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:17.214680 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-614c7b94-362b-4171-90f3-b511c370880b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '080a6a9092fb467eab61323a106a6e01'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:17.215367 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-614c7b94-362b-4171-90f3-b511c370880b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '080a6a9092fb467eab61323a106a6e01'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:17.233810 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1275/2568] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:56:16 2022] DELETE /identity/v3/users/080a6a9092fb467eab61323a106a6e01 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:56:17.680233 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361fcdca-20e7-4b04-ac52-6a41a5692584 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:17.681716 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-361fcdca-20e7-4b04-ac52-6a41a5692584 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:56:17.695648 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361fcdca-20e7-4b04-ac52-6a41a5692584 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:17.726418 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:56:17.752848 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8469710-922d-4408-b9eb-dfedd6b044f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:17.753713 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8469710-922d-4408-b9eb-dfedd6b044f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:17.754004 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8469710-922d-4408-b9eb-dfedd6b044f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:17.754286 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8469710-922d-4408-b9eb-dfedd6b044f9 admin admin] PATH_INFO: `/v3/projects/3bce0ef58c2f4d3cad6fcd3e2d8d9b62` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:17.755480 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8469710-922d-4408-b9eb-dfedd6b044f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bce0ef58c2f4d3cad6fcd3e2d8d9b62)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:17.758003 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8469710-922d-4408-b9eb-dfedd6b044f9 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:17.885294 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f8469710-922d-4408-b9eb-dfedd6b044f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bce0ef58c2f4d3cad6fcd3e2d8d9b62', 'request_id': 'req-f8469710-922d-4408-b9eb-dfedd6b044f9'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:17.958671 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f8469710-922d-4408-b9eb-dfedd6b044f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bce0ef58c2f4d3cad6fcd3e2d8d9b62', 'request_id': 'req-f8469710-922d-4408-b9eb-dfedd6b044f9'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:17.964637 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f8469710-922d-4408-b9eb-dfedd6b044f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bce0ef58c2f4d3cad6fcd3e2d8d9b62', 'request_id': 'req-f8469710-922d-4408-b9eb-dfedd6b044f9'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:17.975747 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f8469710-922d-4408-b9eb-dfedd6b044f9 admin admin] The token cache is being invalidate because project 3bce0ef58c2f4d3cad6fcd3e2d8d9b62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:56:17.976037 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f8469710-922d-4408-b9eb-dfedd6b044f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:17.976668 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f8469710-922d-4408-b9eb-dfedd6b044f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:17.978207 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1294/2569] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:56:17 2022] DELETE /identity/v3/projects/3bce0ef58c2f4d3cad6fcd3e2d8d9b62 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:56:17.983427 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282c6af-cf15-4608-86d5-04785adc257e None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:17.983872 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282c6af-cf15-4608-86d5-04785adc257e None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:17.983972 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282c6af-cf15-4608-86d5-04785adc257e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:18.002701 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-a282c6af-cf15-4608-86d5-04785adc257e None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:56:18.003979 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-a282c6af-cf15-4608-86d5-04785adc257e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:56:18.062823 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1276/2570] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:56:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:56:18.071292 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c4ec6-535f-4f03-891e-83a97e083fba None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:18.071582 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c4ec6-535f-4f03-891e-83a97e083fba None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:18.071994 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c4ec6-535f-4f03-891e-83a97e083fba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:18.090134 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-c88c4ec6-535f-4f03-891e-83a97e083fba None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:56:18.091355 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-c88c4ec6-535f-4f03-891e-83a97e083fba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:56:18.099141 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1295/2571] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:56:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:56:18.103244 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a282c6af-cf15-4608-86d5-04785adc257e None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:18.112780 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a282c6af-cf15-4608-86d5-04785adc257e None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:18.118383 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfee55b-31fb-4cdb-a1cd-8725c6686449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:18.119139 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfee55b-31fb-4cdb-a1cd-8725c6686449 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:18.119369 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfee55b-31fb-4cdb-a1cd-8725c6686449 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:18.119652 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfee55b-31fb-4cdb-a1cd-8725c6686449 admin admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:18.120390 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfee55b-31fb-4cdb-a1cd-8725c6686449 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:56:18.120711 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfee55b-31fb-4cdb-a1cd-8725c6686449 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:18.122340 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfee55b-31fb-4cdb-a1cd-8725c6686449 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:18.134857 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1277/2572] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 2 19:56:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:56:18.139332 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88c4ec6-535f-4f03-891e-83a97e083fba None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:18.148461 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88c4ec6-535f-4f03-891e-83a97e083fba None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:18.153099 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af7436-7f2d-4762-b4c7-11fa734e643e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:18.153746 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af7436-7f2d-4762-b4c7-11fa734e643e admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:18.154034 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af7436-7f2d-4762-b4c7-11fa734e643e admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:18.154314 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af7436-7f2d-4762-b4c7-11fa734e643e admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:18.155043 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71af7436-7f2d-4762-b4c7-11fa734e643e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:18.156953 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71af7436-7f2d-4762-b4c7-11fa734e643e admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:18.267920 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1296/2573] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:56:18 2022] POST /identity/v3/projects => generated 371 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:56:18.272046 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfee55b-31fb-4cdb-a1cd-8725c6686449 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:18.282566 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfee55b-31fb-4cdb-a1cd-8725c6686449 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:18.321270 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:56:18.337274 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060e1ba0-c548-4a9b-bed7-3125dc594593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:18.337938 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060e1ba0-c548-4a9b-bed7-3125dc594593 admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:18.338226 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060e1ba0-c548-4a9b-bed7-3125dc594593 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:18.338506 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060e1ba0-c548-4a9b-bed7-3125dc594593 admin admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:18.339227 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060e1ba0-c548-4a9b-bed7-3125dc594593 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:18.341202 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060e1ba0-c548-4a9b-bed7-3125dc594593 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:18.432673 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1278/2574] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:56:18 2022] POST /identity/v3/users => generated 286 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:56:18.436786 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af7436-7f2d-4762-b4c7-11fa734e643e admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:18.447846 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af7436-7f2d-4762-b4c7-11fa734e643e admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:18.452920 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e27b2a8-ef5b-4840-8a7a-11b48f67c41b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:18.455047 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e27b2a8-ef5b-4840-8a7a-11b48f67c41b admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:18.455047 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e27b2a8-ef5b-4840-8a7a-11b48f67c41b admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:18.455047 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e27b2a8-ef5b-4840-8a7a-11b48f67c41b admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:18.455047 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e27b2a8-ef5b-4840-8a7a-11b48f67c41b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:18.456464 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e27b2a8-ef5b-4840-8a7a-11b48f67c41b admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:18.470549 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1297/2575] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:56:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:56:18.471747 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060e1ba0-c548-4a9b-bed7-3125dc594593 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:18.484280 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060e1ba0-c548-4a9b-bed7-3125dc594593 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:18.489259 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f10768-f7df-45f6-b906-be55e036b645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:18.489913 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f10768-f7df-45f6-b906-be55e036b645 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:18.490196 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f10768-f7df-45f6-b906-be55e036b645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:18.490470 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f10768-f7df-45f6-b906-be55e036b645 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:18.491223 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f10768-f7df-45f6-b906-be55e036b645 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:18.492796 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f10768-f7df-45f6-b906-be55e036b645 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:18.503840 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1279/2576] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:56:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:56:18.508013 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e27b2a8-ef5b-4840-8a7a-11b48f67c41b admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:18.518315 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e27b2a8-ef5b-4840-8a7a-11b48f67c41b admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:18.523539 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e458212-4c40-4dda-9c2b-61123107fcc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:18.524417 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e458212-4c40-4dda-9c2b-61123107fcc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:18.524707 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e458212-4c40-4dda-9c2b-61123107fcc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:18.524982 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e458212-4c40-4dda-9c2b-61123107fcc7 admin admin] PATH_INFO: `/v3/projects/d8ca263ee3f0443a8d59b31b2c32449d/users/674e6df4247b45278d237035f31801e6/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:18.539208 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e458212-4c40-4dda-9c2b-61123107fcc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8ca263ee3f0443a8d59b31b2c32449d, user_id=674e6df4247b45278d237035f31801e6, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:18.541857 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e458212-4c40-4dda-9c2b-61123107fcc7 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:18.644199 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1298/2577] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:56:18 2022] PUT /identity/v3/projects/d8ca263ee3f0443a8d59b31b2c32449d/users/674e6df4247b45278d237035f31801e6/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:56:18.649183 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a0bd43-dc50-4930-874c-57d812755d92 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:18.649483 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a0bd43-dc50-4930-874c-57d812755d92 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:18.649777 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a0bd43-dc50-4930-874c-57d812755d92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:18.672923 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-00a0bd43-dc50-4930-874c-57d812755d92 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:56:18.674172 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-00a0bd43-dc50-4930-874c-57d812755d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:56:18.732677 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1280/2578] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:56:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:56:18.742471 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e458212-4c40-4dda-9c2b-61123107fcc7 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:18.744264 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-8e458212-4c40-4dda-9c2b-61123107fcc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:56:18.809610 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e458212-4c40-4dda-9c2b-61123107fcc7 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:18.813186 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325d9f4e-6414-42ca-a67a-53639855f535 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:18.813834 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d9f4e-6414-42ca-a67a-53639855f535 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:18.814122 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d9f4e-6414-42ca-a67a-53639855f535 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:18.814403 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d9f4e-6414-42ca-a67a-53639855f535 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:18.820912 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:56:18.824471 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325d9f4e-6414-42ca-a67a-53639855f535 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:18.825946 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325d9f4e-6414-42ca-a67a-53639855f535 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:18.835243 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1299/2579] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:56:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:56:18.930300 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a0bd43-dc50-4930-874c-57d812755d92 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:18.931760 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-00a0bd43-dc50-4930-874c-57d812755d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:56:18.992382 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a0bd43-dc50-4930-874c-57d812755d92 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:18.996011 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05af1d15-7f33-4157-898c-b53bf9369edc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:18.996668 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05af1d15-7f33-4157-898c-b53bf9369edc service cinder] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:18.996953 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05af1d15-7f33-4157-898c-b53bf9369edc service cinder] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:18.997230 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05af1d15-7f33-4157-898c-b53bf9369edc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:18.999049 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-05af1d15-7f33-4157-898c-b53bf9369edc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:56:19.070523 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05af1d15-7f33-4157-898c-b53bf9369edc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:19.071932 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05af1d15-7f33-4157-898c-b53bf9369edc service cinder] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:19.081291 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1281/2580] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:56:18 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:56:24.294572 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e376ce-4062-477a-ad1a-5e43b47af596 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:24.295140 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e376ce-4062-477a-ad1a-5e43b47af596 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:24.295140 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e376ce-4062-477a-ad1a-5e43b47af596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:24.323955 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-89e376ce-4062-477a-ad1a-5e43b47af596 None None] MFA Rules not processed for user `674e6df4247b45278d237035f31801e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:56:24.325285 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-89e376ce-4062-477a-ad1a-5e43b47af596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:56:24.397944 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1300/2581] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:56:24 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:56:24.402575 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05af1d15-7f33-4157-898c-b53bf9369edc service cinder] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:24.415301 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05af1d15-7f33-4157-898c-b53bf9369edc service cinder] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:24.416150 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:56:24.420524 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae643c75-8eca-47c5-82fb-887d02010f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:24.421236 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae643c75-8eca-47c5-82fb-887d02010f2c admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:24.421521 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae643c75-8eca-47c5-82fb-887d02010f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:24.421802 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae643c75-8eca-47c5-82fb-887d02010f2c admin admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:24.422684 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae643c75-8eca-47c5-82fb-887d02010f2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:24.424823 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae643c75-8eca-47c5-82fb-887d02010f2c admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:24.543339 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1282/2582] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:56:24 2022] POST /identity/v3/projects => generated 369 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 02 19:56:24.547385 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e376ce-4062-477a-ad1a-5e43b47af596 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:24.558584 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e376ce-4062-477a-ad1a-5e43b47af596 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:24.618968 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3e1d1e-c948-42d1-8753-64348870d230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:24.619699 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3e1d1e-c948-42d1-8753-64348870d230 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:24.619986 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3e1d1e-c948-42d1-8753-64348870d230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:24.620269 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3e1d1e-c948-42d1-8753-64348870d230 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:24.620981 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3e1d1e-c948-42d1-8753-64348870d230 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:24.622995 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3e1d1e-c948-42d1-8753-64348870d230 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:24.701342 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1301/2583] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:56:24 2022] POST /identity/v3/users => generated 291 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:56:24.705369 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae643c75-8eca-47c5-82fb-887d02010f2c admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:24.715697 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae643c75-8eca-47c5-82fb-887d02010f2c admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:24.720667 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310cfec2-a914-434e-a9a7-592efb66afb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:24.721345 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310cfec2-a914-434e-a9a7-592efb66afb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:24.721631 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310cfec2-a914-434e-a9a7-592efb66afb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:24.721913 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310cfec2-a914-434e-a9a7-592efb66afb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:24.722647 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310cfec2-a914-434e-a9a7-592efb66afb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:24.724271 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310cfec2-a914-434e-a9a7-592efb66afb4 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:24.735627 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1283/2584] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:56:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:56:24.739803 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3e1d1e-c948-42d1-8753-64348870d230 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:24.750418 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3e1d1e-c948-42d1-8753-64348870d230 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:24.755144 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccd2c41-2fa8-4c36-8776-6419ecd6ea39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:24.756676 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccd2c41-2fa8-4c36-8776-6419ecd6ea39 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:24.756961 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccd2c41-2fa8-4c36-8776-6419ecd6ea39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:24.757238 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccd2c41-2fa8-4c36-8776-6419ecd6ea39 admin admin] PATH_INFO: `/v3/projects/d4211c56449c4a17af374b99dbf47777/users/e7be4a79a5da472e83f32a112b260794/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:24.772289 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccd2c41-2fa8-4c36-8776-6419ecd6ea39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4211c56449c4a17af374b99dbf47777, user_id=e7be4a79a5da472e83f32a112b260794, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:24.774983 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccd2c41-2fa8-4c36-8776-6419ecd6ea39 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:24.867713 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1302/2585] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:56:24 2022] PUT /identity/v3/projects/d4211c56449c4a17af374b99dbf47777/users/e7be4a79a5da472e83f32a112b260794/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 02 19:56:24.871649 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310cfec2-a914-434e-a9a7-592efb66afb4 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:24.882233 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310cfec2-a914-434e-a9a7-592efb66afb4 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:24.936981 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f1ac5d-ea7f-47d3-b150-1581762bb557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:24.937641 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f1ac5d-ea7f-47d3-b150-1581762bb557 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:24.937931 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f1ac5d-ea7f-47d3-b150-1581762bb557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:24.938213 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f1ac5d-ea7f-47d3-b150-1581762bb557 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:24.938969 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f1ac5d-ea7f-47d3-b150-1581762bb557 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:24.940565 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f1ac5d-ea7f-47d3-b150-1581762bb557 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:24.950717 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1284/2586] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:56:24 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:56:24.954727 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccd2c41-2fa8-4c36-8776-6419ecd6ea39 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:24.965145 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccd2c41-2fa8-4c36-8776-6419ecd6ea39 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:24.969856 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144c5367-0f83-415d-ba94-a6e17968a9c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:24.971072 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144c5367-0f83-415d-ba94-a6e17968a9c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:24.971072 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144c5367-0f83-415d-ba94-a6e17968a9c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:24.971333 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144c5367-0f83-415d-ba94-a6e17968a9c1 admin admin] PATH_INFO: `/v3/projects/d4211c56449c4a17af374b99dbf47777/users/e7be4a79a5da472e83f32a112b260794/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:24.973399 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144c5367-0f83-415d-ba94-a6e17968a9c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4211c56449c4a17af374b99dbf47777, user_id=e7be4a79a5da472e83f32a112b260794, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:24.976627 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144c5367-0f83-415d-ba94-a6e17968a9c1 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:24.991213 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1303/2587] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:56:24 2022] PUT /identity/v3/projects/d4211c56449c4a17af374b99dbf47777/users/e7be4a79a5da472e83f32a112b260794/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:56:25.386842 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f1ac5d-ea7f-47d3-b150-1581762bb557 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:25.386842 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-e6f1ac5d-ea7f-47d3-b150-1581762bb557 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:56:25.400364 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f1ac5d-ea7f-47d3-b150-1581762bb557 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:25.496397 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c9cf85-6196-46f2-be98-f4b473e99944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:25.497884 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c9cf85-6196-46f2-be98-f4b473e99944 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:25.504521 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c9cf85-6196-46f2-be98-f4b473e99944 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:25.504521 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c9cf85-6196-46f2-be98-f4b473e99944 admin admin] PATH_INFO: `/v3/users/77c72931371e4f00a66e848295d70351` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:25.504521 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c9cf85-6196-46f2-be98-f4b473e99944 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77c72931371e4f00a66e848295d70351)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:25.504521 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c9cf85-6196-46f2-be98-f4b473e99944 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:25.706936 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-e7c9cf85-6196-46f2-be98-f4b473e99944 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77c72931371e4f00a66e848295d70351'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:25.718340 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-e7c9cf85-6196-46f2-be98-f4b473e99944 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77c72931371e4f00a66e848295d70351'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:25.719147 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-e7c9cf85-6196-46f2-be98-f4b473e99944 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77c72931371e4f00a66e848295d70351'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:25.802266 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-e7c9cf85-6196-46f2-be98-f4b473e99944 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77c72931371e4f00a66e848295d70351'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:25.803524 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-e7c9cf85-6196-46f2-be98-f4b473e99944 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77c72931371e4f00a66e848295d70351'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:25.809396 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:56:25.822350 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1285/2588] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:56:25 2022] DELETE /identity/v3/users/77c72931371e4f00a66e848295d70351 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:56:25.827002 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d708634-a825-45fb-9697-3056798d3777 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:25.827514 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d708634-a825-45fb-9697-3056798d3777 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:25.827611 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d708634-a825-45fb-9697-3056798d3777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:25.846320 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-4d708634-a825-45fb-9697-3056798d3777 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:56:25.847637 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-4d708634-a825-45fb-9697-3056798d3777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:56:25.883429 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:56:25.924815 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1304/2589] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:56:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:56:25.935862 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c9cf85-6196-46f2-be98-f4b473e99944 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:25.937284 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-e7c9cf85-6196-46f2-be98-f4b473e99944 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:56:25.995338 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c9cf85-6196-46f2-be98-f4b473e99944 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:26.000318 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b13603a-15cb-4178-99f7-7958224d7cf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:26.001038 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b13603a-15cb-4178-99f7-7958224d7cf5 service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:26.006062 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b13603a-15cb-4178-99f7-7958224d7cf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:26.006062 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b13603a-15cb-4178-99f7-7958224d7cf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:26.011346 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b13603a-15cb-4178-99f7-7958224d7cf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:26.012826 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b13603a-15cb-4178-99f7-7958224d7cf5 service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:26.022555 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1286/2590] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:56:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:56:26.491147 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d708634-a825-45fb-9697-3056798d3777 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:26.492656 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-4d708634-a825-45fb-9697-3056798d3777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:56:26.513859 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d708634-a825-45fb-9697-3056798d3777 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:26.520149 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b23b72-de4f-49f4-906d-62149196ed52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:26.521939 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b23b72-de4f-49f4-906d-62149196ed52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:26.522185 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b23b72-de4f-49f4-906d-62149196ed52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:26.525050 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b23b72-de4f-49f4-906d-62149196ed52 admin admin] PATH_INFO: `/v3/projects/7d7a3226e4b8464ea86c242e0d83d975` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:26.526358 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b23b72-de4f-49f4-906d-62149196ed52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d7a3226e4b8464ea86c242e0d83d975)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:26.528408 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b23b72-de4f-49f4-906d-62149196ed52 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:26.697481 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-a5b23b72-de4f-49f4-906d-62149196ed52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d7a3226e4b8464ea86c242e0d83d975', 'request_id': 'req-a5b23b72-de4f-49f4-906d-62149196ed52'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:26.771773 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-a5b23b72-de4f-49f4-906d-62149196ed52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d7a3226e4b8464ea86c242e0d83d975', 'request_id': 'req-a5b23b72-de4f-49f4-906d-62149196ed52'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:26.788122 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-a5b23b72-de4f-49f4-906d-62149196ed52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d7a3226e4b8464ea86c242e0d83d975', 'request_id': 'req-a5b23b72-de4f-49f4-906d-62149196ed52'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:26.804128 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-a5b23b72-de4f-49f4-906d-62149196ed52 admin admin] The token cache is being invalidate because project 7d7a3226e4b8464ea86c242e0d83d975 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:56:26.804430 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-a5b23b72-de4f-49f4-906d-62149196ed52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:26.805152 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-a5b23b72-de4f-49f4-906d-62149196ed52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:26.806897 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1305/2591] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:56:26 2022] DELETE /identity/v3/projects/7d7a3226e4b8464ea86c242e0d83d975 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:56:26.813440 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec982db-2a98-432d-bdcb-88abf89dfbfc None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:26.814453 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec982db-2a98-432d-bdcb-88abf89dfbfc None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:26.815444 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec982db-2a98-432d-bdcb-88abf89dfbfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:26.838667 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-dec982db-2a98-432d-bdcb-88abf89dfbfc None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:56:26.840894 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-dec982db-2a98-432d-bdcb-88abf89dfbfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:56:26.918200 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1287/2592] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:56:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:56:26.926795 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9618caf-dd9a-45e6-9078-46613b08c53a None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:26.928226 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9618caf-dd9a-45e6-9078-46613b08c53a None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:26.928654 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9618caf-dd9a-45e6-9078-46613b08c53a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:26.950560 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-f9618caf-dd9a-45e6-9078-46613b08c53a None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:56:26.952141 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-f9618caf-dd9a-45e6-9078-46613b08c53a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:56:26.962715 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1306/2593] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:56:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:56:26.968222 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec982db-2a98-432d-bdcb-88abf89dfbfc None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:26.979468 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec982db-2a98-432d-bdcb-88abf89dfbfc None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:26.984694 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba16c39c-4f2c-4b14-8c04-af179d055f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:26.985532 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba16c39c-4f2c-4b14-8c04-af179d055f71 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:26.985715 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba16c39c-4f2c-4b14-8c04-af179d055f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:26.985998 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba16c39c-4f2c-4b14-8c04-af179d055f71 admin admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:26.986830 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba16c39c-4f2c-4b14-8c04-af179d055f71 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:56:26.987149 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba16c39c-4f2c-4b14-8c04-af179d055f71 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:26.989107 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba16c39c-4f2c-4b14-8c04-af179d055f71 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:27.013065 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1288/2594] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 2 19:56:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:56:27.017590 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9618caf-dd9a-45e6-9078-46613b08c53a None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:27.036522 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9618caf-dd9a-45e6-9078-46613b08c53a None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:27.045422 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203b6fc1-5cf2-447b-b8be-7ee7444535e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:27.046364 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b6fc1-5cf2-447b-b8be-7ee7444535e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:27.046827 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b6fc1-5cf2-447b-b8be-7ee7444535e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:27.048119 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b6fc1-5cf2-447b-b8be-7ee7444535e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:27.049094 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203b6fc1-5cf2-447b-b8be-7ee7444535e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:27.052084 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203b6fc1-5cf2-447b-b8be-7ee7444535e2 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:27.146673 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1307/2595] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:56:27 2022] POST /identity/v3/projects => generated 373 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:56:27.152498 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba16c39c-4f2c-4b14-8c04-af179d055f71 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:27.172602 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba16c39c-4f2c-4b14-8c04-af179d055f71 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:27.285494 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb739ba-32a9-4dd0-8ca0-ecdab94f7e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:27.286338 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb739ba-32a9-4dd0-8ca0-ecdab94f7e76 admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:27.287413 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb739ba-32a9-4dd0-8ca0-ecdab94f7e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:27.287778 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb739ba-32a9-4dd0-8ca0-ecdab94f7e76 admin admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:27.288502 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb739ba-32a9-4dd0-8ca0-ecdab94f7e76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:27.290651 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb739ba-32a9-4dd0-8ca0-ecdab94f7e76 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:27.394160 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1289/2596] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:56:27 2022] POST /identity/v3/users => generated 287 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:56:27.398791 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203b6fc1-5cf2-447b-b8be-7ee7444535e2 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:27.412889 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203b6fc1-5cf2-447b-b8be-7ee7444535e2 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:27.427073 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b9d8ef-d49f-46be-98ca-19cee1f4af26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:27.428901 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b9d8ef-d49f-46be-98ca-19cee1f4af26 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:27.428901 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b9d8ef-d49f-46be-98ca-19cee1f4af26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:27.428901 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b9d8ef-d49f-46be-98ca-19cee1f4af26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:27.433839 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b9d8ef-d49f-46be-98ca-19cee1f4af26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:27.433839 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b9d8ef-d49f-46be-98ca-19cee1f4af26 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:27.456218 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1308/2597] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:56:27 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:56:27.461250 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb739ba-32a9-4dd0-8ca0-ecdab94f7e76 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:27.474931 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb739ba-32a9-4dd0-8ca0-ecdab94f7e76 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:27.483473 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0ed44c-54db-4268-a8e3-23c062c48fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:27.484475 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ed44c-54db-4268-a8e3-23c062c48fd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:27.485256 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ed44c-54db-4268-a8e3-23c062c48fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:27.485803 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ed44c-54db-4268-a8e3-23c062c48fd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:27.487100 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0ed44c-54db-4268-a8e3-23c062c48fd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:27.491927 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0ed44c-54db-4268-a8e3-23c062c48fd6 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:27.493595 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:56:27.505690 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1290/2598] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:56:27 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:56:27.510604 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b9d8ef-d49f-46be-98ca-19cee1f4af26 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:27.522313 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b9d8ef-d49f-46be-98ca-19cee1f4af26 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:27.527633 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69720e92-62d0-42c3-80de-60c192669c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:27.530630 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69720e92-62d0-42c3-80de-60c192669c3e admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:27.530630 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69720e92-62d0-42c3-80de-60c192669c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:27.530630 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69720e92-62d0-42c3-80de-60c192669c3e admin admin] PATH_INFO: `/v3/projects/5b4afe9f9fe7477894079c2d8f58c277/users/e3decd15017a496f847641fe97a15628/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:27.549892 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69720e92-62d0-42c3-80de-60c192669c3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b4afe9f9fe7477894079c2d8f58c277, user_id=e3decd15017a496f847641fe97a15628, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:27.553359 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69720e92-62d0-42c3-80de-60c192669c3e admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:27.657879 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1309/2599] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:56:27 2022] PUT /identity/v3/projects/5b4afe9f9fe7477894079c2d8f58c277/users/e3decd15017a496f847641fe97a15628/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:56:27.667929 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df15dd71-6f14-4cb3-9b5c-41623fda2481 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:27.670936 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df15dd71-6f14-4cb3-9b5c-41623fda2481 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:27.671525 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df15dd71-6f14-4cb3-9b5c-41623fda2481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:27.716129 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-df15dd71-6f14-4cb3-9b5c-41623fda2481 None None] MFA Rules not processed for user `e3decd15017a496f847641fe97a15628`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:56:27.716129 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-df15dd71-6f14-4cb3-9b5c-41623fda2481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:56:27.806302 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1291/2600] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:56:27 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:56:27.817582 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69720e92-62d0-42c3-80de-60c192669c3e admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:27.819351 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-69720e92-62d0-42c3-80de-60c192669c3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:56:27.882375 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69720e92-62d0-42c3-80de-60c192669c3e admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:27.887072 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6792a0-9909-4eca-8d98-4ec61029f046 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:27.887942 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6792a0-9909-4eca-8d98-4ec61029f046 service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:27.888228 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6792a0-9909-4eca-8d98-4ec61029f046 service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:27.893784 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6792a0-9909-4eca-8d98-4ec61029f046 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:27.899481 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6792a0-9909-4eca-8d98-4ec61029f046 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:27.900950 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6792a0-9909-4eca-8d98-4ec61029f046 service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:27.906477 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:56:27.914945 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1310/2601] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:56:27 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:56:30.862096 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a52dca4-090c-4090-af91-796f4b44b0b5 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:30.862603 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a52dca4-090c-4090-af91-796f4b44b0b5 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:30.862688 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a52dca4-090c-4090-af91-796f4b44b0b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:30.893779 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-2a52dca4-090c-4090-af91-796f4b44b0b5 None None] MFA Rules not processed for user `e7be4a79a5da472e83f32a112b260794`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:56:30.896025 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-2a52dca4-090c-4090-af91-796f4b44b0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:56:31.017540 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1292/2602] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:56:30 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:56:31.035508 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6792a0-9909-4eca-8d98-4ec61029f046 service cinder] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:31.037026 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-4e6792a0-9909-4eca-8d98-4ec61029f046 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:56:31.187196 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6792a0-9909-4eca-8d98-4ec61029f046 service cinder] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:31.199502 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f774a3-e1d7-4d4e-a836-be865f98e300 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:31.200548 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f774a3-e1d7-4d4e-a836-be865f98e300 service nova] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:31.200548 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f774a3-e1d7-4d4e-a836-be865f98e300 service nova] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:31.200548 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f774a3-e1d7-4d4e-a836-be865f98e300 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:31.203194 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-b6f774a3-e1d7-4d4e-a836-be865f98e300 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:56:31.257904 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a52dca4-090c-4090-af91-796f4b44b0b5 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:31.269187 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a52dca4-090c-4090-af91-796f4b44b0b5 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:31.273824 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d6017f-7c31-4bff-867a-81b6525c5b78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:31.274488 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d6017f-7c31-4bff-867a-81b6525c5b78 service cinder] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:31.274824 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d6017f-7c31-4bff-867a-81b6525c5b78 service cinder] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:31.275095 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d6017f-7c31-4bff-867a-81b6525c5b78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:31.276908 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f774a3-e1d7-4d4e-a836-be865f98e300 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:31.277054 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-f8d6017f-7c31-4bff-867a-81b6525c5b78 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:56:31.278525 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f774a3-e1d7-4d4e-a836-be865f98e300 service nova] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:31.289516 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1311/2603] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:56:31 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:56:31.291045 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d6017f-7c31-4bff-867a-81b6525c5b78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:31.292599 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d6017f-7c31-4bff-867a-81b6525c5b78 service cinder] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:31.304293 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1293/2604] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:56:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 02 19:56:32.427385 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f774a3-e1d7-4d4e-a836-be865f98e300 service nova] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:32.491435 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f774a3-e1d7-4d4e-a836-be865f98e300 service nova] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:32.495981 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27e509b-c352-4e9d-9085-bf77541f38e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:32.496848 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27e509b-c352-4e9d-9085-bf77541f38e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:32.497132 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27e509b-c352-4e9d-9085-bf77541f38e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:32.497410 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27e509b-c352-4e9d-9085-bf77541f38e7 admin admin] PATH_INFO: `/v3/users/e3decd15017a496f847641fe97a15628` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:32.498583 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27e509b-c352-4e9d-9085-bf77541f38e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3decd15017a496f847641fe97a15628)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:32.500535 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27e509b-c352-4e9d-9085-bf77541f38e7 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:32.514604 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:56:32.736612 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-d27e509b-c352-4e9d-9085-bf77541f38e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3decd15017a496f847641fe97a15628'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:32.743918 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-d27e509b-c352-4e9d-9085-bf77541f38e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3decd15017a496f847641fe97a15628'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:32.825211 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-d27e509b-c352-4e9d-9085-bf77541f38e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3decd15017a496f847641fe97a15628'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:32.825895 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-d27e509b-c352-4e9d-9085-bf77541f38e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3decd15017a496f847641fe97a15628'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:32.826615 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-d27e509b-c352-4e9d-9085-bf77541f38e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3decd15017a496f847641fe97a15628'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:32.845370 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1312/2605] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:56:32 2022] DELETE /identity/v3/users/e3decd15017a496f847641fe97a15628 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:56:32.849937 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a164c-2ba3-4638-b086-ab0bc252e474 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:32.850414 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a164c-2ba3-4638-b086-ab0bc252e474 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:32.852209 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a164c-2ba3-4638-b086-ab0bc252e474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:32.878424 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-163a164c-2ba3-4638-b086-ab0bc252e474 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:56:32.878424 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-163a164c-2ba3-4638-b086-ab0bc252e474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:56:32.930332 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:56:32.940828 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1294/2606] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:56:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:56:32.951236 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27e509b-c352-4e9d-9085-bf77541f38e7 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:32.952632 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-d27e509b-c352-4e9d-9085-bf77541f38e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:56:33.021820 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27e509b-c352-4e9d-9085-bf77541f38e7 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:33.025741 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ab35e3-dee8-46cc-b405-c7a5d3986ded service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:33.026406 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab35e3-dee8-46cc-b405-c7a5d3986ded service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:33.026698 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab35e3-dee8-46cc-b405-c7a5d3986ded service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:33.027018 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab35e3-dee8-46cc-b405-c7a5d3986ded service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:33.036918 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ab35e3-dee8-46cc-b405-c7a5d3986ded service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:33.040420 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ab35e3-dee8-46cc-b405-c7a5d3986ded service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:33.048064 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1313/2607] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:56:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:56:33.541790 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163a164c-2ba3-4638-b086-ab0bc252e474 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:33.543303 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-163a164c-2ba3-4638-b086-ab0bc252e474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:56:33.555931 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163a164c-2ba3-4638-b086-ab0bc252e474 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:33.561263 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7adb6cb-2c35-40be-a0a5-689c618d75b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:33.562141 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7adb6cb-2c35-40be-a0a5-689c618d75b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:33.562423 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7adb6cb-2c35-40be-a0a5-689c618d75b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:33.565022 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7adb6cb-2c35-40be-a0a5-689c618d75b5 admin admin] PATH_INFO: `/v3/projects/5b4afe9f9fe7477894079c2d8f58c277` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:33.565022 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7adb6cb-2c35-40be-a0a5-689c618d75b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b4afe9f9fe7477894079c2d8f58c277)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:33.565869 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7adb6cb-2c35-40be-a0a5-689c618d75b5 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:33.716497 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-c7adb6cb-2c35-40be-a0a5-689c618d75b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b4afe9f9fe7477894079c2d8f58c277', 'request_id': 'req-c7adb6cb-2c35-40be-a0a5-689c618d75b5'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:33.802626 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-c7adb6cb-2c35-40be-a0a5-689c618d75b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b4afe9f9fe7477894079c2d8f58c277', 'request_id': 'req-c7adb6cb-2c35-40be-a0a5-689c618d75b5'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:33.809592 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-c7adb6cb-2c35-40be-a0a5-689c618d75b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b4afe9f9fe7477894079c2d8f58c277', 'request_id': 'req-c7adb6cb-2c35-40be-a0a5-689c618d75b5'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:33.821535 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-c7adb6cb-2c35-40be-a0a5-689c618d75b5 admin admin] The token cache is being invalidate because project 5b4afe9f9fe7477894079c2d8f58c277 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:56:33.821827 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-c7adb6cb-2c35-40be-a0a5-689c618d75b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:33.822488 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-c7adb6cb-2c35-40be-a0a5-689c618d75b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:33.824058 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1295/2608] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:56:33 2022] DELETE /identity/v3/projects/5b4afe9f9fe7477894079c2d8f58c277 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:56:33.828981 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1912abb3-9a06-4127-87df-ea4a3ee64a88 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:33.829430 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1912abb3-9a06-4127-87df-ea4a3ee64a88 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:33.829548 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1912abb3-9a06-4127-87df-ea4a3ee64a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:33.846855 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-1912abb3-9a06-4127-87df-ea4a3ee64a88 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:56:33.848076 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-1912abb3-9a06-4127-87df-ea4a3ee64a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:56:33.911169 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1314/2609] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:56:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:56:33.918789 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd751ed8-3a34-41e4-99cb-69beb9982cd1 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:33.919257 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd751ed8-3a34-41e4-99cb-69beb9982cd1 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:33.919344 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd751ed8-3a34-41e4-99cb-69beb9982cd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:33.938391 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-dd751ed8-3a34-41e4-99cb-69beb9982cd1 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:56:33.939631 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-dd751ed8-3a34-41e4-99cb-69beb9982cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:56:33.948633 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1296/2610] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:56:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:56:33.952770 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1912abb3-9a06-4127-87df-ea4a3ee64a88 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:33.962865 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1912abb3-9a06-4127-87df-ea4a3ee64a88 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:33.967715 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec58f817-9cf2-4061-b379-e157b6ab10d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:33.968757 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec58f817-9cf2-4061-b379-e157b6ab10d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:33.969051 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec58f817-9cf2-4061-b379-e157b6ab10d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:33.969327 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec58f817-9cf2-4061-b379-e157b6ab10d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:33.970233 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec58f817-9cf2-4061-b379-e157b6ab10d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:56:33.970498 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec58f817-9cf2-4061-b379-e157b6ab10d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:33.972255 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec58f817-9cf2-4061-b379-e157b6ab10d9 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:33.988033 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1315/2611] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 2 19:56:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 19:56:33.992730 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd751ed8-3a34-41e4-99cb-69beb9982cd1 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:34.002629 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd751ed8-3a34-41e4-99cb-69beb9982cd1 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:34.009266 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c56478-262c-4ea6-99c5-c544905ff59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:34.009938 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c56478-262c-4ea6-99c5-c544905ff59a admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:34.010225 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c56478-262c-4ea6-99c5-c544905ff59a admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:34.010506 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c56478-262c-4ea6-99c5-c544905ff59a admin admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:34.011351 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c56478-262c-4ea6-99c5-c544905ff59a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:34.013423 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c56478-262c-4ea6-99c5-c544905ff59a admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:34.128394 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1297/2612] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:56:33 2022] POST /identity/v3/projects => generated 383 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 02 19:56:34.129593 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec58f817-9cf2-4061-b379-e157b6ab10d9 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:34.140268 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec58f817-9cf2-4061-b379-e157b6ab10d9 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:34.155060 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:56:34.200349 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b910cf44-3948-4973-b0ef-07de092af979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:34.201005 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b910cf44-3948-4973-b0ef-07de092af979 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:34.201290 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b910cf44-3948-4973-b0ef-07de092af979 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:34.201565 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b910cf44-3948-4973-b0ef-07de092af979 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:34.202307 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b910cf44-3948-4973-b0ef-07de092af979 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:34.204239 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b910cf44-3948-4973-b0ef-07de092af979 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:34.305790 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1316/2613] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:56:34 2022] POST /identity/v3/users => generated 292 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:56:34.309724 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c56478-262c-4ea6-99c5-c544905ff59a admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:34.320505 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c56478-262c-4ea6-99c5-c544905ff59a admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:34.325258 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef18448-87de-4246-9a1a-0cfebce6e5ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:34.325997 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef18448-87de-4246-9a1a-0cfebce6e5ea admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:34.326372 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef18448-87de-4246-9a1a-0cfebce6e5ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:34.326726 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef18448-87de-4246-9a1a-0cfebce6e5ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:34.327557 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef18448-87de-4246-9a1a-0cfebce6e5ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:34.329226 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef18448-87de-4246-9a1a-0cfebce6e5ea admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:34.343996 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1298/2614] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:56:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:56:34.347625 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba266c98-4863-42a5-a1c8-c6a10c0ff798 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:34.348006 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba266c98-4863-42a5-a1c8-c6a10c0ff798 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:34.348133 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef18448-87de-4246-9a1a-0cfebce6e5ea admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:34.348455 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba266c98-4863-42a5-a1c8-c6a10c0ff798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:34.359062 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef18448-87de-4246-9a1a-0cfebce6e5ea admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:34.364457 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cadd159-e455-49c8-9247-138ee9d85f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:34.370022 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cadd159-e455-49c8-9247-138ee9d85f7f admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:34.370022 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cadd159-e455-49c8-9247-138ee9d85f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:34.370022 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cadd159-e455-49c8-9247-138ee9d85f7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:34.370022 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cadd159-e455-49c8-9247-138ee9d85f7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:34.370022 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cadd159-e455-49c8-9247-138ee9d85f7f admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:34.370562 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-ba266c98-4863-42a5-a1c8-c6a10c0ff798 None None] MFA Rules not processed for user `674e6df4247b45278d237035f31801e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:56:34.370562 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-ba266c98-4863-42a5-a1c8-c6a10c0ff798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:56:34.378418 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1299/2615] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:56:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:56:34.382770 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cadd159-e455-49c8-9247-138ee9d85f7f admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:34.393998 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cadd159-e455-49c8-9247-138ee9d85f7f admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:34.399571 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ee95aa-301f-4b09-9404-9bd426740d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:34.400609 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ee95aa-301f-4b09-9404-9bd426740d4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:34.403840 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ee95aa-301f-4b09-9404-9bd426740d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:34.403840 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ee95aa-301f-4b09-9404-9bd426740d4f admin admin] PATH_INFO: `/v3/projects/401ca64f17cd4c1a999b4650b46c998f/users/f023f0ec06b041b881c00b41bc281299/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:34.418929 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ee95aa-301f-4b09-9404-9bd426740d4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=401ca64f17cd4c1a999b4650b46c998f, user_id=f023f0ec06b041b881c00b41bc281299, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:34.421619 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ee95aa-301f-4b09-9404-9bd426740d4f admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:34.439101 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1317/2616] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:56:34 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:56:34.516358 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1300/2617] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:56:34 2022] PUT /identity/v3/projects/401ca64f17cd4c1a999b4650b46c998f/users/f023f0ec06b041b881c00b41bc281299/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 02 19:56:34.527979 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9de906-177e-4b3a-80a0-5ddd114128c4 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:34.528279 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9de906-177e-4b3a-80a0-5ddd114128c4 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:34.528562 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9de906-177e-4b3a-80a0-5ddd114128c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:34.565675 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-cc9de906-177e-4b3a-80a0-5ddd114128c4 None None] MFA Rules not processed for user `f023f0ec06b041b881c00b41bc281299`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:56:34.566949 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-cc9de906-177e-4b3a-80a0-5ddd114128c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:56:34.642560 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1318/2618] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:56:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:56:34.654187 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ee95aa-301f-4b09-9404-9bd426740d4f admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:34.656083 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-91ee95aa-301f-4b09-9404-9bd426740d4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:56:34.717435 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ee95aa-301f-4b09-9404-9bd426740d4f admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:34.721044 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5cf04b-a974-441d-a203-3e47ce0b0b3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:34.721697 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5cf04b-a974-441d-a203-3e47ce0b0b3c service cinder] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:34.721984 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5cf04b-a974-441d-a203-3e47ce0b0b3c service cinder] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:34.722263 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5cf04b-a974-441d-a203-3e47ce0b0b3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:34.731589 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5cf04b-a974-441d-a203-3e47ce0b0b3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:34.733027 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5cf04b-a974-441d-a203-3e47ce0b0b3c service cinder] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:34.741897 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1301/2619] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:56:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 02 19:56:37.983591 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9de906-177e-4b3a-80a0-5ddd114128c4 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:38.028450 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:56:38.046019 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9de906-177e-4b3a-80a0-5ddd114128c4 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:38.051025 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5634ee5d-9c84-4c4c-a021-7f8193102c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:38.052616 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5634ee5d-9c84-4c4c-a021-7f8193102c75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:38.052903 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5634ee5d-9c84-4c4c-a021-7f8193102c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:38.053184 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5634ee5d-9c84-4c4c-a021-7f8193102c75 admin admin] PATH_INFO: `/v3/users/f023f0ec06b041b881c00b41bc281299` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:38.054400 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5634ee5d-9c84-4c4c-a021-7f8193102c75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f023f0ec06b041b881c00b41bc281299)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:38.056391 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5634ee5d-9c84-4c4c-a021-7f8193102c75 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:38.260290 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-5634ee5d-9c84-4c4c-a021-7f8193102c75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f023f0ec06b041b881c00b41bc281299'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:38.269396 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-5634ee5d-9c84-4c4c-a021-7f8193102c75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f023f0ec06b041b881c00b41bc281299'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:38.355182 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-5634ee5d-9c84-4c4c-a021-7f8193102c75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f023f0ec06b041b881c00b41bc281299'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:38.355826 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-5634ee5d-9c84-4c4c-a021-7f8193102c75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f023f0ec06b041b881c00b41bc281299'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:38.356429 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-5634ee5d-9c84-4c4c-a021-7f8193102c75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f023f0ec06b041b881c00b41bc281299'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:38.375708 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1319/2620] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:56:37 2022] DELETE /identity/v3/users/f023f0ec06b041b881c00b41bc281299 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:56:38.380306 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d028646-7e83-4481-9885-bfde56bb1dd9 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:38.380793 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d028646-7e83-4481-9885-bfde56bb1dd9 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:38.380908 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d028646-7e83-4481-9885-bfde56bb1dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:38.398479 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-7d028646-7e83-4481-9885-bfde56bb1dd9 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:56:38.399735 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-7d028646-7e83-4481-9885-bfde56bb1dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:56:38.405512 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:56:38.460027 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1302/2621] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:56:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:56:38.470106 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5634ee5d-9c84-4c4c-a021-7f8193102c75 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:38.471730 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-5634ee5d-9c84-4c4c-a021-7f8193102c75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:56:38.545011 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5634ee5d-9c84-4c4c-a021-7f8193102c75 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:38.553969 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8b1579-7583-4b30-b7a6-2f76d87476c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:38.553969 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b1579-7583-4b30-b7a6-2f76d87476c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:38.553969 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b1579-7583-4b30-b7a6-2f76d87476c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:38.553969 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b1579-7583-4b30-b7a6-2f76d87476c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:38.562735 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8b1579-7583-4b30-b7a6-2f76d87476c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:38.564191 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8b1579-7583-4b30-b7a6-2f76d87476c2 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:38.574055 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1320/2622] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:56:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:56:38.997310 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d028646-7e83-4481-9885-bfde56bb1dd9 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:38.998769 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-7d028646-7e83-4481-9885-bfde56bb1dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:56:39.011561 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d028646-7e83-4481-9885-bfde56bb1dd9 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:39.016231 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ef05c1-72d5-40b5-9276-3a83f1a85cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:39.017152 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ef05c1-72d5-40b5-9276-3a83f1a85cdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:39.017432 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ef05c1-72d5-40b5-9276-3a83f1a85cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:39.017707 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ef05c1-72d5-40b5-9276-3a83f1a85cdc admin admin] PATH_INFO: `/v3/projects/401ca64f17cd4c1a999b4650b46c998f` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:39.018937 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ef05c1-72d5-40b5-9276-3a83f1a85cdc admin admin] RBAC: Authorizing `identity:delete_project(project_id=401ca64f17cd4c1a999b4650b46c998f)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:39.030221 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ef05c1-72d5-40b5-9276-3a83f1a85cdc admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:39.134145 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-c5ef05c1-72d5-40b5-9276-3a83f1a85cdc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '401ca64f17cd4c1a999b4650b46c998f', 'request_id': 'req-c5ef05c1-72d5-40b5-9276-3a83f1a85cdc'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:39.200562 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-c5ef05c1-72d5-40b5-9276-3a83f1a85cdc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '401ca64f17cd4c1a999b4650b46c998f', 'request_id': 'req-c5ef05c1-72d5-40b5-9276-3a83f1a85cdc'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:39.206713 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-c5ef05c1-72d5-40b5-9276-3a83f1a85cdc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '401ca64f17cd4c1a999b4650b46c998f', 'request_id': 'req-c5ef05c1-72d5-40b5-9276-3a83f1a85cdc'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:39.219776 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-c5ef05c1-72d5-40b5-9276-3a83f1a85cdc admin admin] The token cache is being invalidate because project 401ca64f17cd4c1a999b4650b46c998f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:56:39.220097 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-c5ef05c1-72d5-40b5-9276-3a83f1a85cdc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:39.220761 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-c5ef05c1-72d5-40b5-9276-3a83f1a85cdc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:39.222527 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1303/2623] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:56:38 2022] DELETE /identity/v3/projects/401ca64f17cd4c1a999b4650b46c998f => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:56:39.227987 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92adf25d-53cc-4add-b87a-a920bcc68716 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:39.228463 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92adf25d-53cc-4add-b87a-a920bcc68716 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:39.228568 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92adf25d-53cc-4add-b87a-a920bcc68716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:39.247036 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-92adf25d-53cc-4add-b87a-a920bcc68716 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:56:39.248403 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-92adf25d-53cc-4add-b87a-a920bcc68716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:56:39.312554 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1321/2624] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:56:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:56:39.320185 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b014eab1-94b0-4203-89a3-3da705ce3206 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:39.320482 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b014eab1-94b0-4203-89a3-3da705ce3206 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:39.320763 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b014eab1-94b0-4203-89a3-3da705ce3206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:39.340659 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-b014eab1-94b0-4203-89a3-3da705ce3206 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:56:39.341898 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-b014eab1-94b0-4203-89a3-3da705ce3206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:56:39.350528 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1304/2625] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:56:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:56:39.354773 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92adf25d-53cc-4add-b87a-a920bcc68716 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:39.361358 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:56:39.366342 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92adf25d-53cc-4add-b87a-a920bcc68716 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:39.372031 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bcb531-22b5-4bb8-a403-5b1b8f051fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:39.372362 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bcb531-22b5-4bb8-a403-5b1b8f051fd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:39.372649 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bcb531-22b5-4bb8-a403-5b1b8f051fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:39.372924 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bcb531-22b5-4bb8-a403-5b1b8f051fd1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:39.373662 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bcb531-22b5-4bb8-a403-5b1b8f051fd1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:56:39.373969 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bcb531-22b5-4bb8-a403-5b1b8f051fd1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:39.375605 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bcb531-22b5-4bb8-a403-5b1b8f051fd1 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:39.388319 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1322/2626] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 2 19:56:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:56:39.392245 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b014eab1-94b0-4203-89a3-3da705ce3206 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:39.401523 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b014eab1-94b0-4203-89a3-3da705ce3206 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:39.406227 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b758c869-5c28-479c-b1e3-8bd37e07be4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:39.406916 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b758c869-5c28-479c-b1e3-8bd37e07be4d admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:39.407201 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b758c869-5c28-479c-b1e3-8bd37e07be4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:39.407481 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b758c869-5c28-479c-b1e3-8bd37e07be4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:39.408171 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b758c869-5c28-479c-b1e3-8bd37e07be4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:39.410072 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b758c869-5c28-479c-b1e3-8bd37e07be4d admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:39.517480 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1305/2627] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:56:39 2022] POST /identity/v3/projects => generated 365 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:56:39.521465 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bcb531-22b5-4bb8-a403-5b1b8f051fd1 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:39.532028 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bcb531-22b5-4bb8-a403-5b1b8f051fd1 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:39.606045 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8f1c47-5d98-457d-ae70-393b8f4f1b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:39.607060 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8f1c47-5d98-457d-ae70-393b8f4f1b45 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:39.614203 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8f1c47-5d98-457d-ae70-393b8f4f1b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:39.614203 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8f1c47-5d98-457d-ae70-393b8f4f1b45 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:39.614203 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8f1c47-5d98-457d-ae70-393b8f4f1b45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:39.614203 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8f1c47-5d98-457d-ae70-393b8f4f1b45 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:39.730354 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1323/2628] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:56:39 2022] POST /identity/v3/users => generated 283 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:56:39.736241 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b758c869-5c28-479c-b1e3-8bd37e07be4d admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:39.742220 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b758c869-5c28-479c-b1e3-8bd37e07be4d admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:39.748166 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f63ef84-ed39-4f17-9680-89a59071bd9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:39.748860 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f63ef84-ed39-4f17-9680-89a59071bd9b admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:39.749145 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f63ef84-ed39-4f17-9680-89a59071bd9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:39.749417 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f63ef84-ed39-4f17-9680-89a59071bd9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:39.750136 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f63ef84-ed39-4f17-9680-89a59071bd9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:39.751732 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f63ef84-ed39-4f17-9680-89a59071bd9b admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:39.762447 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1306/2629] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:56:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:56:39.766575 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8f1c47-5d98-457d-ae70-393b8f4f1b45 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:39.778066 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8f1c47-5d98-457d-ae70-393b8f4f1b45 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:39.782740 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddb37b5-4f9c-48c9-a3ba-9700933a0613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:39.783575 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb37b5-4f9c-48c9-a3ba-9700933a0613 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:39.783916 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb37b5-4f9c-48c9-a3ba-9700933a0613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:39.784171 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb37b5-4f9c-48c9-a3ba-9700933a0613 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:39.784921 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddb37b5-4f9c-48c9-a3ba-9700933a0613 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:39.787286 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddb37b5-4f9c-48c9-a3ba-9700933a0613 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:39.799038 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1324/2630] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:56:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:56:39.802685 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f63ef84-ed39-4f17-9680-89a59071bd9b admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:39.812813 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f63ef84-ed39-4f17-9680-89a59071bd9b admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:39.821714 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b1c415-5091-4f28-82ee-fa72cc961c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:39.822620 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b1c415-5091-4f28-82ee-fa72cc961c1a admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:39.822939 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b1c415-5091-4f28-82ee-fa72cc961c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:39.827990 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b1c415-5091-4f28-82ee-fa72cc961c1a admin admin] PATH_INFO: `/v3/projects/4f442cfbb892456790f6899f28e78176/users/953e1a0a0735452db7ae2f7a059f358d/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:39.838474 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b1c415-5091-4f28-82ee-fa72cc961c1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f442cfbb892456790f6899f28e78176, user_id=953e1a0a0735452db7ae2f7a059f358d, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:39.841182 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b1c415-5091-4f28-82ee-fa72cc961c1a admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:39.944883 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1307/2631] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:56:39 2022] PUT /identity/v3/projects/4f442cfbb892456790f6899f28e78176/users/953e1a0a0735452db7ae2f7a059f358d/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:56:39.956254 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3ea8bd-16a3-41f5-a682-3b0a7b623832 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:39.956549 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3ea8bd-16a3-41f5-a682-3b0a7b623832 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:39.956838 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3ea8bd-16a3-41f5-a682-3b0a7b623832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:39.983276 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-fb3ea8bd-16a3-41f5-a682-3b0a7b623832 None None] MFA Rules not processed for user `953e1a0a0735452db7ae2f7a059f358d`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:56:39.989373 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-fb3ea8bd-16a3-41f5-a682-3b0a7b623832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:56:40.057831 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1325/2632] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:56:39 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:56:40.069057 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b1c415-5091-4f28-82ee-fa72cc961c1a admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:40.072250 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-c6b1c415-5091-4f28-82ee-fa72cc961c1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:56:40.103005 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:56:40.129326 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b1c415-5091-4f28-82ee-fa72cc961c1a admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:40.134000 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ededd05-c5ec-4fc4-801a-68ee91bd1e69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:40.134674 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ededd05-c5ec-4fc4-801a-68ee91bd1e69 service cinder] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:40.134995 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ededd05-c5ec-4fc4-801a-68ee91bd1e69 service cinder] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:40.135280 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ededd05-c5ec-4fc4-801a-68ee91bd1e69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:40.144780 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ededd05-c5ec-4fc4-801a-68ee91bd1e69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:40.146319 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ededd05-c5ec-4fc4-801a-68ee91bd1e69 service cinder] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:40.155676 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1308/2633] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:56:40 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:56:50.054837 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=70,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:56:50.234342 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=13 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:56:55.393655 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3ea8bd-16a3-41f5-a682-3b0a7b623832 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:55.395956 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-fb3ea8bd-16a3-41f5-a682-3b0a7b623832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:56:55.453623 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3ea8bd-16a3-41f5-a682-3b0a7b623832 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:55.457986 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61b024b-327a-4766-abc7-e9d6d22ce9ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:55.458638 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61b024b-327a-4766-abc7-e9d6d22ce9ca service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:55.458957 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61b024b-327a-4766-abc7-e9d6d22ce9ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:55.459283 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61b024b-327a-4766-abc7-e9d6d22ce9ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:55.461242 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-f61b024b-327a-4766-abc7-e9d6d22ce9ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:56:55.534955 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61b024b-327a-4766-abc7-e9d6d22ce9ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:55.536464 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61b024b-327a-4766-abc7-e9d6d22ce9ca service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:55.549684 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1326/2634] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:56:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:56:59.457866 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ededd05-c5ec-4fc4-801a-68ee91bd1e69 service cinder] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:59.460345 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-1ededd05-c5ec-4fc4-801a-68ee91bd1e69 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:56:59.536859 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ededd05-c5ec-4fc4-801a-68ee91bd1e69 service cinder] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:59.542216 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c0896c-a581-4b16-bc0e-cb15c3574418 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73ae837e99764bebbe7a6e06ac9049e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:59.542970 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c0896c-a581-4b16-bc0e-cb15c3574418 service placement] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:59.543258 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c0896c-a581-4b16-bc0e-cb15c3574418 service placement] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:59.543722 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c0896c-a581-4b16-bc0e-cb15c3574418 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:59.545493 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-a2c0896c-a581-4b16-bc0e-cb15c3574418 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:56:59.559181 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c0896c-a581-4b16-bc0e-cb15c3574418 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:59.560903 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c0896c-a581-4b16-bc0e-cb15c3574418 service placement] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:59.573402 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1309/2635] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:56:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:56:59.604350 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61b024b-327a-4766-abc7-e9d6d22ce9ca service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:56:59.606225 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-f61b024b-327a-4766-abc7-e9d6d22ce9ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:56:59.638354 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61b024b-327a-4766-abc7-e9d6d22ce9ca service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:56:59.654488 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286c80cd-985f-49fc-b9a2-ce5a2bc82491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:56:59.657366 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c80cd-985f-49fc-b9a2-ce5a2bc82491 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:56:59.657366 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c80cd-985f-49fc-b9a2-ce5a2bc82491 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:56:59.657366 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c80cd-985f-49fc-b9a2-ce5a2bc82491 admin admin] PATH_INFO: `/v3/users/a0681f3eff324262b4289bf6d8f4c311` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:56:59.662141 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286c80cd-985f-49fc-b9a2-ce5a2bc82491 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0681f3eff324262b4289bf6d8f4c311)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:56:59.666152 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286c80cd-985f-49fc-b9a2-ce5a2bc82491 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:56:59.852306 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-286c80cd-985f-49fc-b9a2-ce5a2bc82491 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0681f3eff324262b4289bf6d8f4c311'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:59.859604 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-286c80cd-985f-49fc-b9a2-ce5a2bc82491 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0681f3eff324262b4289bf6d8f4c311'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:59.946045 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-286c80cd-985f-49fc-b9a2-ce5a2bc82491 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0681f3eff324262b4289bf6d8f4c311'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:59.947097 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-286c80cd-985f-49fc-b9a2-ce5a2bc82491 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0681f3eff324262b4289bf6d8f4c311'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:59.947940 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-286c80cd-985f-49fc-b9a2-ce5a2bc82491 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0681f3eff324262b4289bf6d8f4c311'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:56:59.973242 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1327/2636] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:56:59 2022] DELETE /identity/v3/users/a0681f3eff324262b4289bf6d8f4c311 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:57:00.402717 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c0896c-a581-4b16-bc0e-cb15c3574418 service placement] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:57:00.418046 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c0896c-a581-4b16-bc0e-cb15c3574418 service placement] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:57:00.484457 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b475a7-c90a-4719-8190-f91dedcf82dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:57:00.485333 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b475a7-c90a-4719-8190-f91dedcf82dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:00.485621 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b475a7-c90a-4719-8190-f91dedcf82dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:00.485901 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b475a7-c90a-4719-8190-f91dedcf82dc admin admin] PATH_INFO: `/v3/projects/f582628f76674cb9821d1cbf11aaec7d` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:00.487129 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b475a7-c90a-4719-8190-f91dedcf82dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=f582628f76674cb9821d1cbf11aaec7d)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:57:00.489109 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b475a7-c90a-4719-8190-f91dedcf82dc admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:57:00.590868 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-90b475a7-c90a-4719-8190-f91dedcf82dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f582628f76674cb9821d1cbf11aaec7d', 'request_id': 'req-90b475a7-c90a-4719-8190-f91dedcf82dc'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:57:00.653060 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-90b475a7-c90a-4719-8190-f91dedcf82dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f582628f76674cb9821d1cbf11aaec7d', 'request_id': 'req-90b475a7-c90a-4719-8190-f91dedcf82dc'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:57:00.659853 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-90b475a7-c90a-4719-8190-f91dedcf82dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f582628f76674cb9821d1cbf11aaec7d', 'request_id': 'req-90b475a7-c90a-4719-8190-f91dedcf82dc'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:57:00.672302 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-90b475a7-c90a-4719-8190-f91dedcf82dc admin admin] The token cache is being invalidate because project f582628f76674cb9821d1cbf11aaec7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:57:00.672622 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-90b475a7-c90a-4719-8190-f91dedcf82dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:57:00.673377 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-90b475a7-c90a-4719-8190-f91dedcf82dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:57:00.675653 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1310/2637] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:57:00 2022] DELETE /identity/v3/projects/f582628f76674cb9821d1cbf11aaec7d => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:57:00.682396 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b524b352-f7e7-410d-88a1-a62d81584f2d None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:00.682942 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b524b352-f7e7-410d-88a1-a62d81584f2d None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:00.683040 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b524b352-f7e7-410d-88a1-a62d81584f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:00.707307 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-b524b352-f7e7-410d-88a1-a62d81584f2d None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:57:00.708704 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-b524b352-f7e7-410d-88a1-a62d81584f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:57:00.738450 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:57:00.786235 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1328/2638] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:57:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:57:00.794161 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b77b4e3-f55f-4f7a-9e84-9a9700990f86 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:00.794449 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b77b4e3-f55f-4f7a-9e84-9a9700990f86 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:00.794771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b77b4e3-f55f-4f7a-9e84-9a9700990f86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:00.817061 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-8b77b4e3-f55f-4f7a-9e84-9a9700990f86 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:57:00.818324 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-8b77b4e3-f55f-4f7a-9e84-9a9700990f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:57:00.826380 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1311/2639] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:57:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:57:00.830805 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b524b352-f7e7-410d-88a1-a62d81584f2d None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:57:00.840968 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b524b352-f7e7-410d-88a1-a62d81584f2d None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:57:00.845978 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2803030a-2fe6-4d83-bcca-4ec363fb0de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:57:00.846875 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2803030a-2fe6-4d83-bcca-4ec363fb0de7 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:00.847090 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2803030a-2fe6-4d83-bcca-4ec363fb0de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:00.847703 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2803030a-2fe6-4d83-bcca-4ec363fb0de7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:00.848740 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2803030a-2fe6-4d83-bcca-4ec363fb0de7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:57:00.849494 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2803030a-2fe6-4d83-bcca-4ec363fb0de7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:57:00.851244 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2803030a-2fe6-4d83-bcca-4ec363fb0de7 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:57:00.863935 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1329/2640] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 2 19:57:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:57:00.868673 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b77b4e3-f55f-4f7a-9e84-9a9700990f86 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:57:00.879441 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b77b4e3-f55f-4f7a-9e84-9a9700990f86 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:57:00.884513 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf11e57-403f-4fb7-b8fb-bf8d069212f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:57:00.885171 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf11e57-403f-4fb7-b8fb-bf8d069212f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:00.885453 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf11e57-403f-4fb7-b8fb-bf8d069212f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:00.885728 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf11e57-403f-4fb7-b8fb-bf8d069212f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:00.886458 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf11e57-403f-4fb7-b8fb-bf8d069212f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:57:00.888378 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf11e57-403f-4fb7-b8fb-bf8d069212f7 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:57:00.898405 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:57:00.958482 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1312/2641] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:57:00 2022] POST /identity/v3/projects => generated 373 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 02 19:57:00.962557 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2803030a-2fe6-4d83-bcca-4ec363fb0de7 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:57:00.973743 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2803030a-2fe6-4d83-bcca-4ec363fb0de7 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:57:01.040597 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c38d223-7d23-4235-abb2-34024ce30731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:57:01.041305 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c38d223-7d23-4235-abb2-34024ce30731 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:01.041594 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c38d223-7d23-4235-abb2-34024ce30731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:01.041876 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c38d223-7d23-4235-abb2-34024ce30731 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:01.042577 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c38d223-7d23-4235-abb2-34024ce30731 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:57:01.044562 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c38d223-7d23-4235-abb2-34024ce30731 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:57:01.135467 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1330/2642] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:57:00 2022] POST /identity/v3/users => generated 287 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 02 19:57:01.139574 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf11e57-403f-4fb7-b8fb-bf8d069212f7 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:57:01.149993 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf11e57-403f-4fb7-b8fb-bf8d069212f7 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:57:01.154609 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f743a091-5f94-4ae3-ba0a-96df15defaa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:57:01.155485 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f743a091-5f94-4ae3-ba0a-96df15defaa4 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:01.155770 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f743a091-5f94-4ae3-ba0a-96df15defaa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:01.156048 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f743a091-5f94-4ae3-ba0a-96df15defaa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:01.156767 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f743a091-5f94-4ae3-ba0a-96df15defaa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:57:01.158338 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f743a091-5f94-4ae3-ba0a-96df15defaa4 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:57:01.169205 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1313/2643] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:57:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:57:01.173212 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c38d223-7d23-4235-abb2-34024ce30731 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:57:01.184147 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c38d223-7d23-4235-abb2-34024ce30731 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:57:01.188824 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba25b38b-3701-4a0e-85fe-9796f96fe407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:57:01.189496 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25b38b-3701-4a0e-85fe-9796f96fe407 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:01.189786 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25b38b-3701-4a0e-85fe-9796f96fe407 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:01.190067 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25b38b-3701-4a0e-85fe-9796f96fe407 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:01.190839 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba25b38b-3701-4a0e-85fe-9796f96fe407 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:57:01.192450 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba25b38b-3701-4a0e-85fe-9796f96fe407 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:57:01.202426 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1331/2644] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:57:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:57:01.206572 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f743a091-5f94-4ae3-ba0a-96df15defaa4 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:57:01.216684 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f743a091-5f94-4ae3-ba0a-96df15defaa4 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:57:01.221470 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035c02fb-8892-4645-9732-e7961c7d0ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:57:01.223065 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035c02fb-8892-4645-9732-e7961c7d0ccb admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:01.223351 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035c02fb-8892-4645-9732-e7961c7d0ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:01.223628 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035c02fb-8892-4645-9732-e7961c7d0ccb admin admin] PATH_INFO: `/v3/projects/817a761d5bfc43bb970f6d237edd1049/users/09834e38904340e6b2f6c87cdbf2520b/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:01.239386 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035c02fb-8892-4645-9732-e7961c7d0ccb admin admin] RBAC: Authorizing `identity:create_grant(project_id=817a761d5bfc43bb970f6d237edd1049, user_id=09834e38904340e6b2f6c87cdbf2520b, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:57:01.242055 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035c02fb-8892-4645-9732-e7961c7d0ccb admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:57:01.335660 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1314/2645] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:57:01 2022] PUT /identity/v3/projects/817a761d5bfc43bb970f6d237edd1049/users/09834e38904340e6b2f6c87cdbf2520b/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:57:01.345987 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5ac13b-027e-4b41-83e5-b7cb831c328a None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:01.346301 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5ac13b-027e-4b41-83e5-b7cb831c328a None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:01.346583 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5ac13b-027e-4b41-83e5-b7cb831c328a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:01.375653 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-1e5ac13b-027e-4b41-83e5-b7cb831c328a None None] MFA Rules not processed for user `09834e38904340e6b2f6c87cdbf2520b`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:57:01.376878 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-1e5ac13b-027e-4b41-83e5-b7cb831c328a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:57:01.441119 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1332/2646] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:57:01 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:57:01.452743 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035c02fb-8892-4645-9732-e7961c7d0ccb admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:57:01.454495 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-035c02fb-8892-4645-9732-e7961c7d0ccb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:57:01.522878 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035c02fb-8892-4645-9732-e7961c7d0ccb admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:57:01.526548 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4a7b47-13bc-4199-b0fd-62d45e8bd872 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:57:01.527229 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4a7b47-13bc-4199-b0fd-62d45e8bd872 service cinder] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:01.527521 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4a7b47-13bc-4199-b0fd-62d45e8bd872 service cinder] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:01.527802 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4a7b47-13bc-4199-b0fd-62d45e8bd872 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:01.536681 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4a7b47-13bc-4199-b0fd-62d45e8bd872 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:57:01.538110 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4a7b47-13bc-4199-b0fd-62d45e8bd872 service cinder] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:57:01.547771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1315/2647] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:57:01 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:57:01.566886 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5ac13b-027e-4b41-83e5-b7cb831c328a None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:57:01.586256 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:57:01.629574 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5ac13b-027e-4b41-83e5-b7cb831c328a None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:57:01.635155 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b7063-a70b-4626-ad00-3fcc06f6bfea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:57:01.636166 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b7063-a70b-4626-ad00-3fcc06f6bfea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:01.636449 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b7063-a70b-4626-ad00-3fcc06f6bfea admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:01.636726 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b7063-a70b-4626-ad00-3fcc06f6bfea admin admin] PATH_INFO: `/v3/users/09834e38904340e6b2f6c87cdbf2520b` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:01.638010 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0b7063-a70b-4626-ad00-3fcc06f6bfea admin admin] RBAC: Authorizing `identity:delete_user(user_id=09834e38904340e6b2f6c87cdbf2520b)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:57:01.639999 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0b7063-a70b-4626-ad00-3fcc06f6bfea admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:57:01.811186 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-5d0b7063-a70b-4626-ad00-3fcc06f6bfea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09834e38904340e6b2f6c87cdbf2520b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:57:01.826448 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-5d0b7063-a70b-4626-ad00-3fcc06f6bfea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09834e38904340e6b2f6c87cdbf2520b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:57:01.886836 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-5d0b7063-a70b-4626-ad00-3fcc06f6bfea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09834e38904340e6b2f6c87cdbf2520b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:57:01.887465 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-5d0b7063-a70b-4626-ad00-3fcc06f6bfea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09834e38904340e6b2f6c87cdbf2520b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:57:01.888101 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-5d0b7063-a70b-4626-ad00-3fcc06f6bfea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09834e38904340e6b2f6c87cdbf2520b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:57:01.907579 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1333/2648] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:57:01 2022] DELETE /identity/v3/users/09834e38904340e6b2f6c87cdbf2520b => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:57:01.912161 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e757339-dc59-4971-9a13-4cf393477699 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:01.912520 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e757339-dc59-4971-9a13-4cf393477699 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:01.912870 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e757339-dc59-4971-9a13-4cf393477699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:01.929884 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-9e757339-dc59-4971-9a13-4cf393477699 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:57:01.931160 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-9e757339-dc59-4971-9a13-4cf393477699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:57:01.991295 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1316/2649] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:57:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:57:02.001104 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b7063-a70b-4626-ad00-3fcc06f6bfea admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:57:02.002578 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-5d0b7063-a70b-4626-ad00-3fcc06f6bfea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:57:02.060537 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b7063-a70b-4626-ad00-3fcc06f6bfea admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:57:02.064053 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dbc354-685c-4f5a-8ff7-47f43ec30a8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:57:02.064706 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dbc354-685c-4f5a-8ff7-47f43ec30a8e service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:02.064991 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dbc354-685c-4f5a-8ff7-47f43ec30a8e service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:02.065874 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dbc354-685c-4f5a-8ff7-47f43ec30a8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:02.074714 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dbc354-685c-4f5a-8ff7-47f43ec30a8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:57:02.076182 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dbc354-685c-4f5a-8ff7-47f43ec30a8e service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:57:02.086442 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1334/2650] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:57:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:57:02.495949 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e757339-dc59-4971-9a13-4cf393477699 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:57:02.497397 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-9e757339-dc59-4971-9a13-4cf393477699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:57:02.510440 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e757339-dc59-4971-9a13-4cf393477699 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:57:02.515254 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a0769a-c654-4303-aaf1-93af2ef5864c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:57:02.516126 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a0769a-c654-4303-aaf1-93af2ef5864c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:02.516418 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a0769a-c654-4303-aaf1-93af2ef5864c admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:02.516700 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a0769a-c654-4303-aaf1-93af2ef5864c admin admin] PATH_INFO: `/v3/projects/817a761d5bfc43bb970f6d237edd1049` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:02.517930 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a0769a-c654-4303-aaf1-93af2ef5864c admin admin] RBAC: Authorizing `identity:delete_project(project_id=817a761d5bfc43bb970f6d237edd1049)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:57:02.520491 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a0769a-c654-4303-aaf1-93af2ef5864c admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:57:02.616194 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:57:02.618953 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-37a0769a-c654-4303-aaf1-93af2ef5864c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '817a761d5bfc43bb970f6d237edd1049', 'request_id': 'req-37a0769a-c654-4303-aaf1-93af2ef5864c'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:57:02.704072 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-37a0769a-c654-4303-aaf1-93af2ef5864c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '817a761d5bfc43bb970f6d237edd1049', 'request_id': 'req-37a0769a-c654-4303-aaf1-93af2ef5864c'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:57:02.711404 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-37a0769a-c654-4303-aaf1-93af2ef5864c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '817a761d5bfc43bb970f6d237edd1049', 'request_id': 'req-37a0769a-c654-4303-aaf1-93af2ef5864c'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:57:02.722662 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-37a0769a-c654-4303-aaf1-93af2ef5864c admin admin] The token cache is being invalidate because project 817a761d5bfc43bb970f6d237edd1049 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:57:02.722979 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-37a0769a-c654-4303-aaf1-93af2ef5864c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:57:02.723636 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-37a0769a-c654-4303-aaf1-93af2ef5864c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:57:02.725156 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1317/2651] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:57:02 2022] DELETE /identity/v3/projects/817a761d5bfc43bb970f6d237edd1049 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:57:02.730403 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115f9423-55ce-4c97-bbf6-5d9af46d1ce0 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:02.730724 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115f9423-55ce-4c97-bbf6-5d9af46d1ce0 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:02.731032 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115f9423-55ce-4c97-bbf6-5d9af46d1ce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:02.748239 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-115f9423-55ce-4c97-bbf6-5d9af46d1ce0 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:57:02.749442 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-115f9423-55ce-4c97-bbf6-5d9af46d1ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:57:02.786136 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:57:02.826973 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1335/2652] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:57:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:57:02.831196 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc01857-0bd2-450a-aed3-fd5f88b9ff16 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:02.831470 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc01857-0bd2-450a-aed3-fd5f88b9ff16 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:02.831834 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc01857-0bd2-450a-aed3-fd5f88b9ff16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:02.857868 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-2bc01857-0bd2-450a-aed3-fd5f88b9ff16 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:57:02.859199 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-2bc01857-0bd2-450a-aed3-fd5f88b9ff16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:57:02.869135 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1318/2653] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:57:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:57:02.872568 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115f9423-55ce-4c97-bbf6-5d9af46d1ce0 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:57:02.885140 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115f9423-55ce-4c97-bbf6-5d9af46d1ce0 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:57:02.893352 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1676d50b-6a45-4082-be83-8a7b64640d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:57:02.894042 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1676d50b-6a45-4082-be83-8a7b64640d36 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:02.894328 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1676d50b-6a45-4082-be83-8a7b64640d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:02.894608 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1676d50b-6a45-4082-be83-8a7b64640d36 admin admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:02.895386 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1676d50b-6a45-4082-be83-8a7b64640d36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:57:02.895704 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1676d50b-6a45-4082-be83-8a7b64640d36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:57:02.897330 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1676d50b-6a45-4082-be83-8a7b64640d36 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:57:02.913723 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1336/2654] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 2 19:57:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:57:02.915013 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc01857-0bd2-450a-aed3-fd5f88b9ff16 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:57:02.925471 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc01857-0bd2-450a-aed3-fd5f88b9ff16 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:57:02.930290 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da87cf5-88b5-4ae6-a163-dd5a34946ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:57:02.930997 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da87cf5-88b5-4ae6-a163-dd5a34946ffc admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:02.931349 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da87cf5-88b5-4ae6-a163-dd5a34946ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:02.931629 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da87cf5-88b5-4ae6-a163-dd5a34946ffc admin admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:02.932326 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da87cf5-88b5-4ae6-a163-dd5a34946ffc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:57:02.934320 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da87cf5-88b5-4ae6-a163-dd5a34946ffc admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:57:03.010389 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1319/2655] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:57:02 2022] POST /identity/v3/projects => generated 373 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:57:03.014973 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1676d50b-6a45-4082-be83-8a7b64640d36 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:57:03.025388 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1676d50b-6a45-4082-be83-8a7b64640d36 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:57:03.084083 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb66164-94e1-4a6e-8333-b618b1b51990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:57:03.084755 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb66164-94e1-4a6e-8333-b618b1b51990 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:03.085040 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb66164-94e1-4a6e-8333-b618b1b51990 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:03.085315 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb66164-94e1-4a6e-8333-b618b1b51990 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:03.086011 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb66164-94e1-4a6e-8333-b618b1b51990 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:57:03.088009 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb66164-94e1-4a6e-8333-b618b1b51990 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:57:03.164220 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1337/2656] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:57:03 2022] POST /identity/v3/users => generated 287 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:57:03.168172 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da87cf5-88b5-4ae6-a163-dd5a34946ffc admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:57:03.179527 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da87cf5-88b5-4ae6-a163-dd5a34946ffc admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:57:03.184235 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ab20dd-8388-4c12-97b7-2953f4aab5fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:57:03.184921 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab20dd-8388-4c12-97b7-2953f4aab5fa admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:03.185205 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab20dd-8388-4c12-97b7-2953f4aab5fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:03.185482 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab20dd-8388-4c12-97b7-2953f4aab5fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:03.186204 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ab20dd-8388-4c12-97b7-2953f4aab5fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:57:03.188017 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ab20dd-8388-4c12-97b7-2953f4aab5fa admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:57:03.198444 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1320/2657] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:57:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:57:03.202354 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb66164-94e1-4a6e-8333-b618b1b51990 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:57:03.213293 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb66164-94e1-4a6e-8333-b618b1b51990 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:57:03.218004 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a52e8d-f26d-45f6-9e1c-aa452310dffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:57:03.218681 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a52e8d-f26d-45f6-9e1c-aa452310dffa admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:03.219009 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a52e8d-f26d-45f6-9e1c-aa452310dffa admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:03.219296 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a52e8d-f26d-45f6-9e1c-aa452310dffa admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:03.220032 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a52e8d-f26d-45f6-9e1c-aa452310dffa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:57:03.223272 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a52e8d-f26d-45f6-9e1c-aa452310dffa admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:57:03.232463 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1338/2658] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:57:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:57:03.236470 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ab20dd-8388-4c12-97b7-2953f4aab5fa admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:57:03.247116 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ab20dd-8388-4c12-97b7-2953f4aab5fa admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:57:03.251768 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1cd206-4d45-405c-b2e8-273c76645a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:57:03.252945 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1cd206-4d45-405c-b2e8-273c76645a0e admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:03.253225 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1cd206-4d45-405c-b2e8-273c76645a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:03.253499 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1cd206-4d45-405c-b2e8-273c76645a0e admin admin] PATH_INFO: `/v3/projects/a72646b2e4cd478bb41acaf7332aa01b/users/7a8b93c910b34c54a32e8d879262f9e3/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:03.269148 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1cd206-4d45-405c-b2e8-273c76645a0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a72646b2e4cd478bb41acaf7332aa01b, user_id=7a8b93c910b34c54a32e8d879262f9e3, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:57:03.271904 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1cd206-4d45-405c-b2e8-273c76645a0e admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:57:03.351397 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1321/2659] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:57:03 2022] PUT /identity/v3/projects/a72646b2e4cd478bb41acaf7332aa01b/users/7a8b93c910b34c54a32e8d879262f9e3/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:57:03.410460 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba5c688-65db-4b58-87da-efb3b65b30ba None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:03.410769 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba5c688-65db-4b58-87da-efb3b65b30ba None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:03.411067 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba5c688-65db-4b58-87da-efb3b65b30ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:03.438042 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-bba5c688-65db-4b58-87da-efb3b65b30ba None None] MFA Rules not processed for user `7a8b93c910b34c54a32e8d879262f9e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:57:03.439213 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-bba5c688-65db-4b58-87da-efb3b65b30ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:57:03.504361 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1339/2660] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:57:03 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:57:03.515382 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1cd206-4d45-405c-b2e8-273c76645a0e admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:57:03.517186 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-1c1cd206-4d45-405c-b2e8-273c76645a0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:57:03.614424 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1cd206-4d45-405c-b2e8-273c76645a0e admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:57:03.618380 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639712e5-cb1d-4641-bff3-931bc5dc1289 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07892c0aa21449b94a1cc78e3b8a653', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:57:03.619087 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639712e5-cb1d-4641-bff3-931bc5dc1289 service glance] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:03.619377 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639712e5-cb1d-4641-bff3-931bc5dc1289 service glance] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:03.619658 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639712e5-cb1d-4641-bff3-931bc5dc1289 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:03.630511 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639712e5-cb1d-4641-bff3-931bc5dc1289 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:57:03.632019 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639712e5-cb1d-4641-bff3-931bc5dc1289 service glance] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:57:03.641712 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1322/2661] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:57:03 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 02 19:57:13.500833 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=81,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:57:13.637459 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:DELETE=4,keystone:SELECT=69,keystone:INSERT=3 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:57:14.884019 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba5c688-65db-4b58-87da-efb3b65b30ba None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:57:14.885460 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-bba5c688-65db-4b58-87da-efb3b65b30ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:57:14.960984 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba5c688-65db-4b58-87da-efb3b65b30ba None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:57:14.964955 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd351bd2-3ff5-4ce2-83c1-fc9725d33c5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:57:14.965645 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd351bd2-3ff5-4ce2-83c1-fc9725d33c5b service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:14.965932 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd351bd2-3ff5-4ce2-83c1-fc9725d33c5b service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:14.966213 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd351bd2-3ff5-4ce2-83c1-fc9725d33c5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:14.968088 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-fd351bd2-3ff5-4ce2-83c1-fc9725d33c5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:57:15.030047 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd351bd2-3ff5-4ce2-83c1-fc9725d33c5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:57:15.031513 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd351bd2-3ff5-4ce2-83c1-fc9725d33c5b service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:57:15.041028 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1340/2662] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:57:14 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:57:24.548158 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639712e5-cb1d-4641-bff3-931bc5dc1289 service glance] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:57:24.549615 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-639712e5-cb1d-4641-bff3-931bc5dc1289 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:57:24.620246 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639712e5-cb1d-4641-bff3-931bc5dc1289 service glance] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:57:24.625055 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6521481-8ab1-43c7-b2bd-808cde6acda6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:57:24.625938 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6521481-8ab1-43c7-b2bd-808cde6acda6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:24.626223 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6521481-8ab1-43c7-b2bd-808cde6acda6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:24.626501 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6521481-8ab1-43c7-b2bd-808cde6acda6 admin admin] PATH_INFO: `/v3/users/953e1a0a0735452db7ae2f7a059f358d` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:24.627749 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6521481-8ab1-43c7-b2bd-808cde6acda6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=953e1a0a0735452db7ae2f7a059f358d)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:57:24.629636 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6521481-8ab1-43c7-b2bd-808cde6acda6 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:57:24.829348 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-e6521481-8ab1-43c7-b2bd-808cde6acda6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '953e1a0a0735452db7ae2f7a059f358d'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:57:24.836445 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-e6521481-8ab1-43c7-b2bd-808cde6acda6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '953e1a0a0735452db7ae2f7a059f358d'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:57:24.837221 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-e6521481-8ab1-43c7-b2bd-808cde6acda6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '953e1a0a0735452db7ae2f7a059f358d'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:57:24.923852 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-e6521481-8ab1-43c7-b2bd-808cde6acda6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '953e1a0a0735452db7ae2f7a059f358d'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:57:24.924462 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-e6521481-8ab1-43c7-b2bd-808cde6acda6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '953e1a0a0735452db7ae2f7a059f358d'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:57:24.942944 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1323/2663] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:57:24 2022] DELETE /identity/v3/users/953e1a0a0735452db7ae2f7a059f358d => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:57:24.947401 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5551e80a-8766-4875-956c-83edbdb798fa None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:24.947881 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5551e80a-8766-4875-956c-83edbdb798fa None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:24.947981 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5551e80a-8766-4875-956c-83edbdb798fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:24.965931 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-5551e80a-8766-4875-956c-83edbdb798fa None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:57:24.967173 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-5551e80a-8766-4875-956c-83edbdb798fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:57:25.036129 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1341/2664] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:57:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:57:25.045668 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6521481-8ab1-43c7-b2bd-808cde6acda6 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:57:25.047092 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-e6521481-8ab1-43c7-b2bd-808cde6acda6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:57:25.103422 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6521481-8ab1-43c7-b2bd-808cde6acda6 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:57:25.107044 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d386eb08-33c8-4715-9e4f-662d04826bf9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:57:25.107692 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d386eb08-33c8-4715-9e4f-662d04826bf9 service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:25.107978 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d386eb08-33c8-4715-9e4f-662d04826bf9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:25.108259 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d386eb08-33c8-4715-9e4f-662d04826bf9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:25.118182 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d386eb08-33c8-4715-9e4f-662d04826bf9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:57:25.119748 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d386eb08-33c8-4715-9e4f-662d04826bf9 service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:57:25.130733 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1324/2665] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:57:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:57:25.531150 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5551e80a-8766-4875-956c-83edbdb798fa None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:57:25.532611 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-5551e80a-8766-4875-956c-83edbdb798fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:57:25.545982 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5551e80a-8766-4875-956c-83edbdb798fa None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:57:25.550742 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c1874d-06ce-4ff6-be20-f53b5e57abbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:57:25.551722 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c1874d-06ce-4ff6-be20-f53b5e57abbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:25.552007 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c1874d-06ce-4ff6-be20-f53b5e57abbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:25.552282 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c1874d-06ce-4ff6-be20-f53b5e57abbe admin admin] PATH_INFO: `/v3/projects/4f442cfbb892456790f6899f28e78176` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:25.553477 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c1874d-06ce-4ff6-be20-f53b5e57abbe admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f442cfbb892456790f6899f28e78176)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:57:25.555420 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c1874d-06ce-4ff6-be20-f53b5e57abbe admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:57:25.657310 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:57:25.667093 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-39c1874d-06ce-4ff6-be20-f53b5e57abbe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f442cfbb892456790f6899f28e78176', 'request_id': 'req-39c1874d-06ce-4ff6-be20-f53b5e57abbe'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:57:25.735001 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-39c1874d-06ce-4ff6-be20-f53b5e57abbe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f442cfbb892456790f6899f28e78176', 'request_id': 'req-39c1874d-06ce-4ff6-be20-f53b5e57abbe'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:57:25.741262 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-39c1874d-06ce-4ff6-be20-f53b5e57abbe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f442cfbb892456790f6899f28e78176', 'request_id': 'req-39c1874d-06ce-4ff6-be20-f53b5e57abbe'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:57:25.752569 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-39c1874d-06ce-4ff6-be20-f53b5e57abbe admin admin] The token cache is being invalidate because project 4f442cfbb892456790f6899f28e78176 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:57:25.752861 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-39c1874d-06ce-4ff6-be20-f53b5e57abbe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:57:25.753488 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-39c1874d-06ce-4ff6-be20-f53b5e57abbe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:57:25.755112 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1342/2666] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:57:25 2022] DELETE /identity/v3/projects/4f442cfbb892456790f6899f28e78176 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:57:25.759998 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d1d80-907b-475d-b5c1-d5ea97e1118d None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:25.760442 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d1d80-907b-475d-b5c1-d5ea97e1118d None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:25.760572 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d1d80-907b-475d-b5c1-d5ea97e1118d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:25.777996 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-c98d1d80-907b-475d-b5c1-d5ea97e1118d None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:57:25.779240 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-c98d1d80-907b-475d-b5c1-d5ea97e1118d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:57:25.796465 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:57:25.837978 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1325/2667] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:57:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:57:25.845980 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c1dac-76d5-4f48-8d5c-5944436721ff None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:25.846377 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c1dac-76d5-4f48-8d5c-5944436721ff None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:25.846707 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c1dac-76d5-4f48-8d5c-5944436721ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:25.867217 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-185c1dac-76d5-4f48-8d5c-5944436721ff None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:57:25.868413 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-185c1dac-76d5-4f48-8d5c-5944436721ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:57:25.877050 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1343/2668] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:57:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:57:25.881281 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98d1d80-907b-475d-b5c1-d5ea97e1118d None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:57:25.890788 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98d1d80-907b-475d-b5c1-d5ea97e1118d None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:57:25.895923 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfc48c7-79e0-4454-b07e-90bd7f780461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:57:25.896572 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfc48c7-79e0-4454-b07e-90bd7f780461 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:25.896858 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfc48c7-79e0-4454-b07e-90bd7f780461 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:25.897135 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfc48c7-79e0-4454-b07e-90bd7f780461 admin admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:25.897862 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfc48c7-79e0-4454-b07e-90bd7f780461 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:57:25.898169 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfc48c7-79e0-4454-b07e-90bd7f780461 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:57:25.899817 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfc48c7-79e0-4454-b07e-90bd7f780461 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:57:25.916139 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1326/2669] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 2 19:57:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:57:25.916295 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185c1dac-76d5-4f48-8d5c-5944436721ff None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:57:25.925367 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185c1dac-76d5-4f48-8d5c-5944436721ff None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:57:25.929957 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4906e6-99b3-404a-9ea9-49c7fa257f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:57:25.930597 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4906e6-99b3-404a-9ea9-49c7fa257f1d admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:25.930921 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4906e6-99b3-404a-9ea9-49c7fa257f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:25.931195 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4906e6-99b3-404a-9ea9-49c7fa257f1d admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:25.931878 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4906e6-99b3-404a-9ea9-49c7fa257f1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:57:25.933787 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4906e6-99b3-404a-9ea9-49c7fa257f1d admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:57:26.011017 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1344/2670] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:57:25 2022] POST /identity/v3/projects => generated 375 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:57:26.015477 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfc48c7-79e0-4454-b07e-90bd7f780461 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:57:26.026408 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfc48c7-79e0-4454-b07e-90bd7f780461 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:57:26.065960 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4906e6-99b3-404a-9ea9-49c7fa257f1d admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:57:26.076016 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-4f4906e6-99b3-404a-9ea9-49c7fa257f1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:57:26.091437 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15317599-7d4d-4c26-8221-f9799174e894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:57:26.092177 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15317599-7d4d-4c26-8221-f9799174e894 admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:26.092469 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15317599-7d4d-4c26-8221-f9799174e894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:26.092752 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15317599-7d4d-4c26-8221-f9799174e894 admin admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:26.093474 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15317599-7d4d-4c26-8221-f9799174e894 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:57:26.095571 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15317599-7d4d-4c26-8221-f9799174e894 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:57:26.146461 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4906e6-99b3-404a-9ea9-49c7fa257f1d admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:57:26.151166 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2d25bb-3ac6-423c-bb96-50b07bae5b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:57:26.152068 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2d25bb-3ac6-423c-bb96-50b07bae5b2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:26.152358 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2d25bb-3ac6-423c-bb96-50b07bae5b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:26.152637 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2d25bb-3ac6-423c-bb96-50b07bae5b2c admin admin] PATH_INFO: `/v3/users/7a8b93c910b34c54a32e8d879262f9e3` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:26.153849 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2d25bb-3ac6-423c-bb96-50b07bae5b2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a8b93c910b34c54a32e8d879262f9e3)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:57:26.155809 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2d25bb-3ac6-423c-bb96-50b07bae5b2c admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:57:26.187315 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1327/2671] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:57:26 2022] POST /identity/v3/users => generated 288 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:57:26.191248 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15317599-7d4d-4c26-8221-f9799174e894 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:57:26.201240 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15317599-7d4d-4c26-8221-f9799174e894 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:57:26.205886 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2be764d-e013-458c-b71d-2212aa76326f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:57:26.206525 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be764d-e013-458c-b71d-2212aa76326f admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:26.206856 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be764d-e013-458c-b71d-2212aa76326f admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:26.207143 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be764d-e013-458c-b71d-2212aa76326f admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:26.208397 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2be764d-e013-458c-b71d-2212aa76326f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:57:26.209975 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2be764d-e013-458c-b71d-2212aa76326f admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:57:26.219908 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1328/2672] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:57:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 02 19:57:26.224102 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2be764d-e013-458c-b71d-2212aa76326f admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:57:26.233193 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2be764d-e013-458c-b71d-2212aa76326f admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:57:26.237781 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c16087f-9e62-4e45-8de1-59dab090e613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:57:26.238376 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c16087f-9e62-4e45-8de1-59dab090e613 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:26.238710 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c16087f-9e62-4e45-8de1-59dab090e613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:26.239023 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c16087f-9e62-4e45-8de1-59dab090e613 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:26.239717 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c16087f-9e62-4e45-8de1-59dab090e613 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:57:26.241220 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c16087f-9e62-4e45-8de1-59dab090e613 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:57:26.251984 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1329/2673] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:57:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 02 19:57:26.256422 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c16087f-9e62-4e45-8de1-59dab090e613 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:57:26.265856 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c16087f-9e62-4e45-8de1-59dab090e613 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:57:26.271009 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517c4118-91ed-4efa-a5f5-543066b89d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:57:26.271867 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c4118-91ed-4efa-a5f5-543066b89d87 admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:26.272148 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c4118-91ed-4efa-a5f5-543066b89d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:26.272428 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c4118-91ed-4efa-a5f5-543066b89d87 admin admin] PATH_INFO: `/v3/projects/e984a07eacdb4f63965a5c8c340743af/users/6aa3d0de3377412ca332c8568c21102a/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:26.286537 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517c4118-91ed-4efa-a5f5-543066b89d87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e984a07eacdb4f63965a5c8c340743af, user_id=6aa3d0de3377412ca332c8568c21102a, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:57:26.289185 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517c4118-91ed-4efa-a5f5-543066b89d87 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:57:26.338238 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-1a2d25bb-3ac6-423c-bb96-50b07bae5b2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a8b93c910b34c54a32e8d879262f9e3'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:57:26.345049 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-1a2d25bb-3ac6-423c-bb96-50b07bae5b2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a8b93c910b34c54a32e8d879262f9e3'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:57:26.369780 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1330/2674] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:57:26 2022] PUT /identity/v3/projects/e984a07eacdb4f63965a5c8c340743af/users/6aa3d0de3377412ca332c8568c21102a/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 02 19:57:26.380874 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7ae47a-7cd8-4932-907f-e99a43aaa1f9 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:26.381167 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7ae47a-7cd8-4932-907f-e99a43aaa1f9 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:26.381449 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7ae47a-7cd8-4932-907f-e99a43aaa1f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:26.408114 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-bd7ae47a-7cd8-4932-907f-e99a43aaa1f9 None None] MFA Rules not processed for user `6aa3d0de3377412ca332c8568c21102a`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:57:26.409322 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-bd7ae47a-7cd8-4932-907f-e99a43aaa1f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:57:26.414242 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-1a2d25bb-3ac6-423c-bb96-50b07bae5b2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a8b93c910b34c54a32e8d879262f9e3'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:57:26.414948 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-1a2d25bb-3ac6-423c-bb96-50b07bae5b2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a8b93c910b34c54a32e8d879262f9e3'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:57:26.415574 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-1a2d25bb-3ac6-423c-bb96-50b07bae5b2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a8b93c910b34c54a32e8d879262f9e3'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:57:26.434401 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1345/2675] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:57:26 2022] DELETE /identity/v3/users/7a8b93c910b34c54a32e8d879262f9e3 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 02 19:57:26.438911 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce7be6e-85c8-4145-af2a-efd9381c034d None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:26.439150 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce7be6e-85c8-4145-af2a-efd9381c034d None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:26.439433 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce7be6e-85c8-4145-af2a-efd9381c034d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:26.451705 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:57:26.458112 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-5ce7be6e-85c8-4145-af2a-efd9381c034d None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:57:26.460111 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-5ce7be6e-85c8-4145-af2a-efd9381c034d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:57:26.468357 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:DELETE=15,keystone:SELECT=82,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:57:26.474765 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1331/2676] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:57:26 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 19:57:26.485439 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7ae47a-7cd8-4932-907f-e99a43aaa1f9 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:57:26.486869 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-bd7ae47a-7cd8-4932-907f-e99a43aaa1f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:57:26.522127 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1346/2677] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:57:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 19:57:26.531633 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce7be6e-85c8-4145-af2a-efd9381c034d None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:57:26.533060 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-5ce7be6e-85c8-4145-af2a-efd9381c034d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:57:26.544020 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7ae47a-7cd8-4932-907f-e99a43aaa1f9 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:57:26.590380 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce7be6e-85c8-4145-af2a-efd9381c034d None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:57:26.593661 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aee1f76-2787-4254-85ec-0c07d37ece88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:57:26.594166 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e840312-6e24-4de9-8ba9-0bb64dd83cc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:57:26.594310 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee1f76-2787-4254-85ec-0c07d37ece88 service cinder] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:26.594576 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee1f76-2787-4254-85ec-0c07d37ece88 service cinder] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:26.594926 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e840312-6e24-4de9-8ba9-0bb64dd83cc1 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:26.595029 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee1f76-2787-4254-85ec-0c07d37ece88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:26.595217 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e840312-6e24-4de9-8ba9-0bb64dd83cc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:26.595598 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e840312-6e24-4de9-8ba9-0bb64dd83cc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:26.606698 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e840312-6e24-4de9-8ba9-0bb64dd83cc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:57:26.608184 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e840312-6e24-4de9-8ba9-0bb64dd83cc1 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:57:26.618530 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1347/2678] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:57:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 02 19:57:26.652924 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aee1f76-2787-4254-85ec-0c07d37ece88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:57:26.654365 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aee1f76-2787-4254-85ec-0c07d37ece88 service cinder] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:57:26.664087 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1332/2679] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:57:26 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 02 19:57:27.067947 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e840312-6e24-4de9-8ba9-0bb64dd83cc1 service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:57:27.070644 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-6e840312-6e24-4de9-8ba9-0bb64dd83cc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:57:27.082808 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e840312-6e24-4de9-8ba9-0bb64dd83cc1 service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:57:27.087497 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5c62e3-3172-49d8-bffe-aa2a16314d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:57:27.088499 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c62e3-3172-49d8-bffe-aa2a16314d5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:27.088772 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c62e3-3172-49d8-bffe-aa2a16314d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:27.089066 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c62e3-3172-49d8-bffe-aa2a16314d5e admin admin] PATH_INFO: `/v3/projects/a72646b2e4cd478bb41acaf7332aa01b` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:27.090251 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5c62e3-3172-49d8-bffe-aa2a16314d5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a72646b2e4cd478bb41acaf7332aa01b)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:57:27.092203 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5c62e3-3172-49d8-bffe-aa2a16314d5e admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:57:27.215953 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-5d5c62e3-3172-49d8-bffe-aa2a16314d5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a72646b2e4cd478bb41acaf7332aa01b', 'request_id': 'req-5d5c62e3-3172-49d8-bffe-aa2a16314d5e'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:57:27.305781 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-5d5c62e3-3172-49d8-bffe-aa2a16314d5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a72646b2e4cd478bb41acaf7332aa01b', 'request_id': 'req-5d5c62e3-3172-49d8-bffe-aa2a16314d5e'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:57:27.313465 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-5d5c62e3-3172-49d8-bffe-aa2a16314d5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a72646b2e4cd478bb41acaf7332aa01b', 'request_id': 'req-5d5c62e3-3172-49d8-bffe-aa2a16314d5e'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:57:27.335263 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-5d5c62e3-3172-49d8-bffe-aa2a16314d5e admin admin] The token cache is being invalidate because project a72646b2e4cd478bb41acaf7332aa01b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:57:27.335263 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-5d5c62e3-3172-49d8-bffe-aa2a16314d5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:57:27.335263 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-5d5c62e3-3172-49d8-bffe-aa2a16314d5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:57:27.335263 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1348/2680] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:57:27 2022] DELETE /identity/v3/projects/a72646b2e4cd478bb41acaf7332aa01b => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:57:27.336309 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce5cfa4-635c-40e3-9e55-c1a27c7263d0 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:27.336774 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce5cfa4-635c-40e3-9e55-c1a27c7263d0 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:27.336943 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce5cfa4-635c-40e3-9e55-c1a27c7263d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:27.355251 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-2ce5cfa4-635c-40e3-9e55-c1a27c7263d0 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:57:27.356543 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-2ce5cfa4-635c-40e3-9e55-c1a27c7263d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:57:27.405459 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:57:27.417489 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1333/2681] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:57:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:57:27.425227 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90155b95-3c65-4c35-af29-f4fab723e036 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:27.425594 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90155b95-3c65-4c35-af29-f4fab723e036 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:27.427012 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90155b95-3c65-4c35-af29-f4fab723e036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:27.447089 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-90155b95-3c65-4c35-af29-f4fab723e036 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:57:27.447570 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-90155b95-3c65-4c35-af29-f4fab723e036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:57:27.455837 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1349/2682] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:57:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:57:27.460487 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce5cfa4-635c-40e3-9e55-c1a27c7263d0 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:57:27.470626 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce5cfa4-635c-40e3-9e55-c1a27c7263d0 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:57:27.476344 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3191d3e0-88af-4cb8-8c42-3e48b6dadc15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:57:27.477017 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3191d3e0-88af-4cb8-8c42-3e48b6dadc15 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:27.477305 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3191d3e0-88af-4cb8-8c42-3e48b6dadc15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:27.477584 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3191d3e0-88af-4cb8-8c42-3e48b6dadc15 admin admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:27.478324 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3191d3e0-88af-4cb8-8c42-3e48b6dadc15 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:57:27.478643 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3191d3e0-88af-4cb8-8c42-3e48b6dadc15 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:57:27.480283 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3191d3e0-88af-4cb8-8c42-3e48b6dadc15 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:57:27.492089 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1334/2683] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 2 19:57:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:57:27.496480 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90155b95-3c65-4c35-af29-f4fab723e036 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:57:27.506111 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90155b95-3c65-4c35-af29-f4fab723e036 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:57:27.510889 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1881a31-0294-40a0-8c34-7c3ea002254b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:57:27.511536 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1881a31-0294-40a0-8c34-7c3ea002254b admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:27.511822 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1881a31-0294-40a0-8c34-7c3ea002254b admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:27.512100 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1881a31-0294-40a0-8c34-7c3ea002254b admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:27.512800 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1881a31-0294-40a0-8c34-7c3ea002254b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:57:27.514716 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1881a31-0294-40a0-8c34-7c3ea002254b admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:57:27.615932 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1350/2684] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:57:27 2022] POST /identity/v3/projects => generated 371 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:57:27.620070 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3191d3e0-88af-4cb8-8c42-3e48b6dadc15 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:57:27.630545 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3191d3e0-88af-4cb8-8c42-3e48b6dadc15 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:57:27.686003 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02808f51-c93a-4c2f-b3f1-e8649cc34107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:57:27.686667 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02808f51-c93a-4c2f-b3f1-e8649cc34107 admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:27.687020 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02808f51-c93a-4c2f-b3f1-e8649cc34107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:27.687293 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02808f51-c93a-4c2f-b3f1-e8649cc34107 admin admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:27.687987 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02808f51-c93a-4c2f-b3f1-e8649cc34107 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:57:27.689874 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02808f51-c93a-4c2f-b3f1-e8649cc34107 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:57:27.781179 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1335/2685] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:57:27 2022] POST /identity/v3/users => generated 286 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:57:27.786507 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1881a31-0294-40a0-8c34-7c3ea002254b admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:57:27.796871 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1881a31-0294-40a0-8c34-7c3ea002254b admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:57:27.802051 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cd108c-4df8-4a1c-92b6-0b9609c087cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:57:27.802724 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cd108c-4df8-4a1c-92b6-0b9609c087cd admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:27.803035 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cd108c-4df8-4a1c-92b6-0b9609c087cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:27.803317 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cd108c-4df8-4a1c-92b6-0b9609c087cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:27.804058 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cd108c-4df8-4a1c-92b6-0b9609c087cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:57:27.805670 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cd108c-4df8-4a1c-92b6-0b9609c087cd admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:57:27.810995 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:57:27.816392 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1351/2686] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:57:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:57:27.820193 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02808f51-c93a-4c2f-b3f1-e8649cc34107 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:57:27.830158 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02808f51-c93a-4c2f-b3f1-e8649cc34107 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:57:27.835167 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe5f1bf-7cce-4760-8b95-1bfb3e1b33ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:57:27.835861 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe5f1bf-7cce-4760-8b95-1bfb3e1b33ff admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:27.836207 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe5f1bf-7cce-4760-8b95-1bfb3e1b33ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:27.836493 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe5f1bf-7cce-4760-8b95-1bfb3e1b33ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:27.837224 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe5f1bf-7cce-4760-8b95-1bfb3e1b33ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:57:27.845166 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe5f1bf-7cce-4760-8b95-1bfb3e1b33ff admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:57:27.855864 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1336/2687] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:57:27 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:57:27.860056 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cd108c-4df8-4a1c-92b6-0b9609c087cd admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:57:27.870635 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cd108c-4df8-4a1c-92b6-0b9609c087cd admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:57:27.875648 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313ee90d-1b38-493d-af81-406cd79b5b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:57:27.876599 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ee90d-1b38-493d-af81-406cd79b5b79 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:27.876883 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ee90d-1b38-493d-af81-406cd79b5b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:27.877158 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ee90d-1b38-493d-af81-406cd79b5b79 admin admin] PATH_INFO: `/v3/projects/75ea22c055ba46c199dc52727dbc2fd7/users/0a59b08b7df949b9a41b9a3f5e1d8d5e/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:27.891456 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313ee90d-1b38-493d-af81-406cd79b5b79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75ea22c055ba46c199dc52727dbc2fd7, user_id=0a59b08b7df949b9a41b9a3f5e1d8d5e, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:57:27.894174 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313ee90d-1b38-493d-af81-406cd79b5b79 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:57:27.993072 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1352/2688] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:57:27 2022] PUT /identity/v3/projects/75ea22c055ba46c199dc52727dbc2fd7/users/0a59b08b7df949b9a41b9a3f5e1d8d5e/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:57:28.005109 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c93001b-41da-4ba3-bb5a-baece36748d4 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:28.005399 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c93001b-41da-4ba3-bb5a-baece36748d4 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:28.005702 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c93001b-41da-4ba3-bb5a-baece36748d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:28.032587 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-3c93001b-41da-4ba3-bb5a-baece36748d4 None None] MFA Rules not processed for user `0a59b08b7df949b9a41b9a3f5e1d8d5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:57:28.033784 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-3c93001b-41da-4ba3-bb5a-baece36748d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:57:28.096312 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1337/2689] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:57:28 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:57:28.107634 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313ee90d-1b38-493d-af81-406cd79b5b79 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:57:28.109421 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-313ee90d-1b38-493d-af81-406cd79b5b79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:57:28.168471 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313ee90d-1b38-493d-af81-406cd79b5b79 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:57:28.172228 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2784f6fd-7409-4c25-977a-588febd53797 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07892c0aa21449b94a1cc78e3b8a653', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:57:28.172881 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2784f6fd-7409-4c25-977a-588febd53797 service glance] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:28.173171 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2784f6fd-7409-4c25-977a-588febd53797 service glance] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:28.173458 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2784f6fd-7409-4c25-977a-588febd53797 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:28.182657 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2784f6fd-7409-4c25-977a-588febd53797 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:57:28.184132 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2784f6fd-7409-4c25-977a-588febd53797 service glance] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:57:28.193721 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1353/2690] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:57:28 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:57:37.877614 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c93001b-41da-4ba3-bb5a-baece36748d4 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:57:37.879108 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-3c93001b-41da-4ba3-bb5a-baece36748d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:57:37.929128 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:57:37.945840 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c93001b-41da-4ba3-bb5a-baece36748d4 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:57:37.950651 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb495b75-976a-4d36-8a19-f347f2d22334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:57:37.951567 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb495b75-976a-4d36-8a19-f347f2d22334 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:37.951853 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb495b75-976a-4d36-8a19-f347f2d22334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:37.952130 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb495b75-976a-4d36-8a19-f347f2d22334 admin admin] PATH_INFO: `/v3/users/6aa3d0de3377412ca332c8568c21102a` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:37.953323 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb495b75-976a-4d36-8a19-f347f2d22334 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6aa3d0de3377412ca332c8568c21102a)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:57:37.955364 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb495b75-976a-4d36-8a19-f347f2d22334 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:57:38.127301 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-fb495b75-976a-4d36-8a19-f347f2d22334 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6aa3d0de3377412ca332c8568c21102a'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:57:38.135090 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-fb495b75-976a-4d36-8a19-f347f2d22334 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6aa3d0de3377412ca332c8568c21102a'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:57:38.135797 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-fb495b75-976a-4d36-8a19-f347f2d22334 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6aa3d0de3377412ca332c8568c21102a'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:57:38.189533 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=36,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:57:38.202351 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-fb495b75-976a-4d36-8a19-f347f2d22334 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6aa3d0de3377412ca332c8568c21102a'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:57:38.203011 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-fb495b75-976a-4d36-8a19-f347f2d22334 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6aa3d0de3377412ca332c8568c21102a'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:57:38.221967 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1338/2691] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:57:37 2022] DELETE /identity/v3/users/6aa3d0de3377412ca332c8568c21102a => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:57:38.227019 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3332f83-4d2f-4dd6-83dc-d7efe443603b None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:38.227309 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3332f83-4d2f-4dd6-83dc-d7efe443603b None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:38.227589 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3332f83-4d2f-4dd6-83dc-d7efe443603b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:38.245972 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-d3332f83-4d2f-4dd6-83dc-d7efe443603b None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:57:38.247466 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-d3332f83-4d2f-4dd6-83dc-d7efe443603b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:57:38.307853 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1354/2692] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:57:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:57:38.317412 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb495b75-976a-4d36-8a19-f347f2d22334 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:57:38.318914 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-fb495b75-976a-4d36-8a19-f347f2d22334 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:57:38.378781 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb495b75-976a-4d36-8a19-f347f2d22334 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:57:38.382418 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa29aa-7b74-4297-86f4-2b101d3968d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:57:38.383109 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa29aa-7b74-4297-86f4-2b101d3968d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:38.383400 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa29aa-7b74-4297-86f4-2b101d3968d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:38.383684 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa29aa-7b74-4297-86f4-2b101d3968d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:38.392811 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aa29aa-7b74-4297-86f4-2b101d3968d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:57:38.394235 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aa29aa-7b74-4297-86f4-2b101d3968d4 service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:57:38.403718 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1339/2693] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:57:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:57:38.787438 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3332f83-4d2f-4dd6-83dc-d7efe443603b None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:57:38.789728 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-d3332f83-4d2f-4dd6-83dc-d7efe443603b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:57:38.802308 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3332f83-4d2f-4dd6-83dc-d7efe443603b None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:57:38.807034 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2ec065-81f2-4a5b-91dd-b5d2f2b510d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:57:38.807998 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ec065-81f2-4a5b-91dd-b5d2f2b510d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:38.808359 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ec065-81f2-4a5b-91dd-b5d2f2b510d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:38.808717 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ec065-81f2-4a5b-91dd-b5d2f2b510d3 admin admin] PATH_INFO: `/v3/projects/e984a07eacdb4f63965a5c8c340743af` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:38.809976 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2ec065-81f2-4a5b-91dd-b5d2f2b510d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e984a07eacdb4f63965a5c8c340743af)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:57:38.812203 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2ec065-81f2-4a5b-91dd-b5d2f2b510d3 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:57:38.933773 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-6e2ec065-81f2-4a5b-91dd-b5d2f2b510d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e984a07eacdb4f63965a5c8c340743af', 'request_id': 'req-6e2ec065-81f2-4a5b-91dd-b5d2f2b510d3'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:57:39.013664 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-6e2ec065-81f2-4a5b-91dd-b5d2f2b510d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e984a07eacdb4f63965a5c8c340743af', 'request_id': 'req-6e2ec065-81f2-4a5b-91dd-b5d2f2b510d3'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:57:39.020049 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-6e2ec065-81f2-4a5b-91dd-b5d2f2b510d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e984a07eacdb4f63965a5c8c340743af', 'request_id': 'req-6e2ec065-81f2-4a5b-91dd-b5d2f2b510d3'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:57:39.032124 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-6e2ec065-81f2-4a5b-91dd-b5d2f2b510d3 admin admin] The token cache is being invalidate because project e984a07eacdb4f63965a5c8c340743af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:57:39.032416 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-6e2ec065-81f2-4a5b-91dd-b5d2f2b510d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:57:39.033050 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-6e2ec065-81f2-4a5b-91dd-b5d2f2b510d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:57:39.034792 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1355/2694] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:57:38 2022] DELETE /identity/v3/projects/e984a07eacdb4f63965a5c8c340743af => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 02 19:57:48.399474 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=57,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:57:49.029000 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:57:58.584664 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa29aa-7b74-4297-86f4-2b101d3968d4 service neutron] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:57:58.586063 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-58aa29aa-7b74-4297-86f4-2b101d3968d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:57:58.598770 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa29aa-7b74-4297-86f4-2b101d3968d4 service neutron] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:57:58.655990 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0064d278-7a84-4729-bf75-e3fd623af18f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:57:58.656858 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0064d278-7a84-4729-bf75-e3fd623af18f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:58.657151 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0064d278-7a84-4729-bf75-e3fd623af18f admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:58.657432 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0064d278-7a84-4729-bf75-e3fd623af18f admin admin] PATH_INFO: `/v3/users/0a59b08b7df949b9a41b9a3f5e1d8d5e` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:58.658623 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0064d278-7a84-4729-bf75-e3fd623af18f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a59b08b7df949b9a41b9a3f5e1d8d5e)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:57:58.660599 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0064d278-7a84-4729-bf75-e3fd623af18f admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:57:58.868000 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-0064d278-7a84-4729-bf75-e3fd623af18f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a59b08b7df949b9a41b9a3f5e1d8d5e'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:57:58.874966 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-0064d278-7a84-4729-bf75-e3fd623af18f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a59b08b7df949b9a41b9a3f5e1d8d5e'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:57:58.875664 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-0064d278-7a84-4729-bf75-e3fd623af18f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a59b08b7df949b9a41b9a3f5e1d8d5e'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:57:58.939967 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-0064d278-7a84-4729-bf75-e3fd623af18f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a59b08b7df949b9a41b9a3f5e1d8d5e'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:57:58.940582 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-0064d278-7a84-4729-bf75-e3fd623af18f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a59b08b7df949b9a41b9a3f5e1d8d5e'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:57:58.958214 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1340/2695] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:57:58 2022] DELETE /identity/v3/users/0a59b08b7df949b9a41b9a3f5e1d8d5e => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:57:58.963280 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28af38f-a24e-4d9b-87a4-e6a2666881fd None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:58.963727 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28af38f-a24e-4d9b-87a4-e6a2666881fd None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:58.963838 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28af38f-a24e-4d9b-87a4-e6a2666881fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:58.981752 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-f28af38f-a24e-4d9b-87a4-e6a2666881fd None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:57:58.982966 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-f28af38f-a24e-4d9b-87a4-e6a2666881fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:57:59.040027 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1356/2696] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:57:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:57:59.050066 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0064d278-7a84-4729-bf75-e3fd623af18f admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:57:59.051500 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-0064d278-7a84-4729-bf75-e3fd623af18f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:57:59.107734 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0064d278-7a84-4729-bf75-e3fd623af18f admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:57:59.111308 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc35c887-4e42-4dfc-9c36-39a6279b00b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:57:59.111960 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc35c887-4e42-4dfc-9c36-39a6279b00b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:59.112249 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc35c887-4e42-4dfc-9c36-39a6279b00b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:59.112582 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc35c887-4e42-4dfc-9c36-39a6279b00b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:59.122216 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc35c887-4e42-4dfc-9c36-39a6279b00b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:57:59.123627 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc35c887-4e42-4dfc-9c36-39a6279b00b1 service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:57:59.133609 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1341/2697] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:57:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:57:59.524364 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28af38f-a24e-4d9b-87a4-e6a2666881fd None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:57:59.525818 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-f28af38f-a24e-4d9b-87a4-e6a2666881fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:57:59.537829 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28af38f-a24e-4d9b-87a4-e6a2666881fd None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:57:59.542449 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27378698-5118-4dbf-b5b8-d52790749930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:57:59.543353 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27378698-5118-4dbf-b5b8-d52790749930 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:59.543635 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27378698-5118-4dbf-b5b8-d52790749930 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:59.543910 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27378698-5118-4dbf-b5b8-d52790749930 admin admin] PATH_INFO: `/v3/projects/75ea22c055ba46c199dc52727dbc2fd7` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:59.545115 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27378698-5118-4dbf-b5b8-d52790749930 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75ea22c055ba46c199dc52727dbc2fd7)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:57:59.547066 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27378698-5118-4dbf-b5b8-d52790749930 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:57:59.643781 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-27378698-5118-4dbf-b5b8-d52790749930 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75ea22c055ba46c199dc52727dbc2fd7', 'request_id': 'req-27378698-5118-4dbf-b5b8-d52790749930'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:57:59.695268 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-27378698-5118-4dbf-b5b8-d52790749930 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75ea22c055ba46c199dc52727dbc2fd7', 'request_id': 'req-27378698-5118-4dbf-b5b8-d52790749930'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:57:59.700923 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-27378698-5118-4dbf-b5b8-d52790749930 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75ea22c055ba46c199dc52727dbc2fd7', 'request_id': 'req-27378698-5118-4dbf-b5b8-d52790749930'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:57:59.711824 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-27378698-5118-4dbf-b5b8-d52790749930 admin admin] The token cache is being invalidate because project 75ea22c055ba46c199dc52727dbc2fd7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:57:59.712109 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-27378698-5118-4dbf-b5b8-d52790749930 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:57:59.712747 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-27378698-5118-4dbf-b5b8-d52790749930 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:57:59.714298 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1357/2698] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:57:59 2022] DELETE /identity/v3/projects/75ea22c055ba46c199dc52727dbc2fd7 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:57:59.719298 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe2606-b07e-4e3c-8199-8aae4b0c1270 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:59.719787 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe2606-b07e-4e3c-8199-8aae4b0c1270 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:59.719883 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe2606-b07e-4e3c-8199-8aae4b0c1270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:59.737170 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-27fe2606-b07e-4e3c-8199-8aae4b0c1270 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:57:59.738377 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-27fe2606-b07e-4e3c-8199-8aae4b0c1270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:57:59.757131 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:57:59.798554 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1342/2699] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:57:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:57:59.806858 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7b5ba0-9a69-4831-9846-736cd4404678 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:59.807177 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7b5ba0-9a69-4831-9846-736cd4404678 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:59.807463 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7b5ba0-9a69-4831-9846-736cd4404678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:59.826106 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-9c7b5ba0-9a69-4831-9846-736cd4404678 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:57:59.827345 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-9c7b5ba0-9a69-4831-9846-736cd4404678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:57:59.835185 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1358/2700] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:57:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:57:59.839315 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fe2606-b07e-4e3c-8199-8aae4b0c1270 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:57:59.848870 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fe2606-b07e-4e3c-8199-8aae4b0c1270 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:57:59.853987 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae68064-20e4-4c9e-a05c-a1c46757d171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:57:59.854675 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae68064-20e4-4c9e-a05c-a1c46757d171 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:59.855206 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae68064-20e4-4c9e-a05c-a1c46757d171 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:59.855715 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae68064-20e4-4c9e-a05c-a1c46757d171 admin admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:59.856828 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae68064-20e4-4c9e-a05c-a1c46757d171 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:57:59.857144 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae68064-20e4-4c9e-a05c-a1c46757d171 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:57:59.858735 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae68064-20e4-4c9e-a05c-a1c46757d171 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:57:59.871098 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1343/2701] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 2 19:57:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:57:59.875233 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7b5ba0-9a69-4831-9846-736cd4404678 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:57:59.884208 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7b5ba0-9a69-4831-9846-736cd4404678 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:57:59.888689 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84dbebd-66ac-45ac-a545-d709912c6634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:57:59.889331 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84dbebd-66ac-45ac-a545-d709912c6634 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:57:59.889610 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84dbebd-66ac-45ac-a545-d709912c6634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:57:59.889884 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84dbebd-66ac-45ac-a545-d709912c6634 admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:57:59.890570 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84dbebd-66ac-45ac-a545-d709912c6634 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:57:59.892479 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84dbebd-66ac-45ac-a545-d709912c6634 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:57:59.965507 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1359/2702] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:57:59 2022] POST /identity/v3/projects => generated 373 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:57:59.972230 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae68064-20e4-4c9e-a05c-a1c46757d171 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:57:59.982604 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae68064-20e4-4c9e-a05c-a1c46757d171 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:58:00.038244 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78bda8a-ea33-4757-b9fc-51813ca2aaad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:58:00.038951 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78bda8a-ea33-4757-b9fc-51813ca2aaad admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:58:00.039240 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78bda8a-ea33-4757-b9fc-51813ca2aaad admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:58:00.039532 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78bda8a-ea33-4757-b9fc-51813ca2aaad admin admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:58:00.040227 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78bda8a-ea33-4757-b9fc-51813ca2aaad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:58:00.042135 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78bda8a-ea33-4757-b9fc-51813ca2aaad admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:58:00.095038 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1344/2703] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:57:59 2022] POST /identity/v3/users => generated 287 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:58:00.099068 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84dbebd-66ac-45ac-a545-d709912c6634 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:58:00.110063 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84dbebd-66ac-45ac-a545-d709912c6634 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:58:00.114943 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19850eab-179c-4b71-990e-cf36e690eb6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:58:00.116212 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19850eab-179c-4b71-990e-cf36e690eb6f admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:58:00.116500 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19850eab-179c-4b71-990e-cf36e690eb6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:58:00.116783 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19850eab-179c-4b71-990e-cf36e690eb6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:58:00.117533 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19850eab-179c-4b71-990e-cf36e690eb6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:58:00.119542 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19850eab-179c-4b71-990e-cf36e690eb6f admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:58:00.131326 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1360/2704] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:58:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:58:00.135228 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78bda8a-ea33-4757-b9fc-51813ca2aaad admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:58:00.145040 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78bda8a-ea33-4757-b9fc-51813ca2aaad admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:58:00.149704 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97614af6-b153-4070-a843-76ee74902ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:58:00.150363 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97614af6-b153-4070-a843-76ee74902ef7 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:58:00.150651 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97614af6-b153-4070-a843-76ee74902ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:58:00.150967 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97614af6-b153-4070-a843-76ee74902ef7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:58:00.151703 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97614af6-b153-4070-a843-76ee74902ef7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:58:00.153321 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97614af6-b153-4070-a843-76ee74902ef7 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:58:00.163649 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1345/2705] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:58:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:58:00.167516 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19850eab-179c-4b71-990e-cf36e690eb6f admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:58:00.177381 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19850eab-179c-4b71-990e-cf36e690eb6f admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:58:00.182641 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65622e2-368e-40ee-8a81-8094e546525f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:58:00.183562 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65622e2-368e-40ee-8a81-8094e546525f admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:58:00.183833 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65622e2-368e-40ee-8a81-8094e546525f admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:58:00.184107 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65622e2-368e-40ee-8a81-8094e546525f admin admin] PATH_INFO: `/v3/projects/cf7a2c8c93344099b70e14e2837a62be/users/85618772831f40b58414424ba8b7dfda/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:58:00.198220 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65622e2-368e-40ee-8a81-8094e546525f admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf7a2c8c93344099b70e14e2837a62be, user_id=85618772831f40b58414424ba8b7dfda, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:58:00.201209 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65622e2-368e-40ee-8a81-8094e546525f admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:58:00.278733 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1361/2706] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:58:00 2022] PUT /identity/v3/projects/cf7a2c8c93344099b70e14e2837a62be/users/85618772831f40b58414424ba8b7dfda/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:58:00.289263 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443fca3e-87ea-426b-a7ad-579df4245136 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:58:00.289561 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443fca3e-87ea-426b-a7ad-579df4245136 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:58:00.289842 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443fca3e-87ea-426b-a7ad-579df4245136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:58:00.315664 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-443fca3e-87ea-426b-a7ad-579df4245136 None None] MFA Rules not processed for user `85618772831f40b58414424ba8b7dfda`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:58:00.316905 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-443fca3e-87ea-426b-a7ad-579df4245136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:58:00.379831 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1346/2707] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:58:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:58:00.392286 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65622e2-368e-40ee-8a81-8094e546525f admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:58:00.394164 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-d65622e2-368e-40ee-8a81-8094e546525f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:58:00.416898 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:58:00.451194 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65622e2-368e-40ee-8a81-8094e546525f admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:58:00.454868 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b26f1ce-db12-45b0-8045-af40d65665f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:58:00.455515 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b26f1ce-db12-45b0-8045-af40d65665f9 service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:58:00.457403 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b26f1ce-db12-45b0-8045-af40d65665f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:58:00.457403 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b26f1ce-db12-45b0-8045-af40d65665f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:58:00.464790 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b26f1ce-db12-45b0-8045-af40d65665f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:58:00.466226 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b26f1ce-db12-45b0-8045-af40d65665f9 service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:58:00.475958 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1362/2708] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:58:00 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:58:00.499702 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443fca3e-87ea-426b-a7ad-579df4245136 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:58:00.518770 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:58:00.561597 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443fca3e-87ea-426b-a7ad-579df4245136 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:58:00.566608 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f71ac0-6ad4-4340-9fcc-67087d965b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:58:00.567509 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f71ac0-6ad4-4340-9fcc-67087d965b66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:58:00.567792 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f71ac0-6ad4-4340-9fcc-67087d965b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:58:00.568098 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f71ac0-6ad4-4340-9fcc-67087d965b66 admin admin] PATH_INFO: `/v3/users/85618772831f40b58414424ba8b7dfda` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:58:00.569844 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f71ac0-6ad4-4340-9fcc-67087d965b66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85618772831f40b58414424ba8b7dfda)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:58:00.571804 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f71ac0-6ad4-4340-9fcc-67087d965b66 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:58:00.747615 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-a0f71ac0-6ad4-4340-9fcc-67087d965b66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85618772831f40b58414424ba8b7dfda'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:58:00.754376 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-a0f71ac0-6ad4-4340-9fcc-67087d965b66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85618772831f40b58414424ba8b7dfda'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:58:00.755146 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-a0f71ac0-6ad4-4340-9fcc-67087d965b66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85618772831f40b58414424ba8b7dfda'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:58:00.825639 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-a0f71ac0-6ad4-4340-9fcc-67087d965b66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85618772831f40b58414424ba8b7dfda'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:58:00.826388 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-a0f71ac0-6ad4-4340-9fcc-67087d965b66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85618772831f40b58414424ba8b7dfda'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:58:00.845339 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1347/2709] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:58:00 2022] DELETE /identity/v3/users/85618772831f40b58414424ba8b7dfda => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:58:00.849807 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7252de0b-9ce2-4e17-b570-2e5eb93cf8bc None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:58:00.850100 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7252de0b-9ce2-4e17-b570-2e5eb93cf8bc None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:58:00.852577 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7252de0b-9ce2-4e17-b570-2e5eb93cf8bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:58:00.866857 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-7252de0b-9ce2-4e17-b570-2e5eb93cf8bc None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:58:00.868031 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-7252de0b-9ce2-4e17-b570-2e5eb93cf8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:58:00.926371 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1363/2710] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:58:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:58:00.935571 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f71ac0-6ad4-4340-9fcc-67087d965b66 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:58:00.937003 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-a0f71ac0-6ad4-4340-9fcc-67087d965b66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:58:00.994780 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f71ac0-6ad4-4340-9fcc-67087d965b66 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:58:00.998506 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db3856c-aabc-40db-9d7b-c037cbddeeb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:58:00.999344 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db3856c-aabc-40db-9d7b-c037cbddeeb1 service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:58:00.999628 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db3856c-aabc-40db-9d7b-c037cbddeeb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:58:00.999906 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db3856c-aabc-40db-9d7b-c037cbddeeb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:58:01.009984 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db3856c-aabc-40db-9d7b-c037cbddeeb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:58:01.012868 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db3856c-aabc-40db-9d7b-c037cbddeeb1 service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:58:01.021072 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1348/2711] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:58:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:58:01.382796 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7252de0b-9ce2-4e17-b570-2e5eb93cf8bc None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:58:01.384289 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-7252de0b-9ce2-4e17-b570-2e5eb93cf8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:58:01.396221 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7252de0b-9ce2-4e17-b570-2e5eb93cf8bc None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:58:01.400681 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010eacb1-7af7-4068-889d-9b8ddfd25c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:58:01.401607 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010eacb1-7af7-4068-889d-9b8ddfd25c38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:58:01.401889 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010eacb1-7af7-4068-889d-9b8ddfd25c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:58:01.402167 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010eacb1-7af7-4068-889d-9b8ddfd25c38 admin admin] PATH_INFO: `/v3/projects/cf7a2c8c93344099b70e14e2837a62be` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:58:01.403382 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010eacb1-7af7-4068-889d-9b8ddfd25c38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf7a2c8c93344099b70e14e2837a62be)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:58:01.405277 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010eacb1-7af7-4068-889d-9b8ddfd25c38 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:58:01.539793 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-010eacb1-7af7-4068-889d-9b8ddfd25c38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf7a2c8c93344099b70e14e2837a62be', 'request_id': 'req-010eacb1-7af7-4068-889d-9b8ddfd25c38'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:58:01.613763 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-010eacb1-7af7-4068-889d-9b8ddfd25c38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf7a2c8c93344099b70e14e2837a62be', 'request_id': 'req-010eacb1-7af7-4068-889d-9b8ddfd25c38'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:58:01.620455 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-010eacb1-7af7-4068-889d-9b8ddfd25c38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf7a2c8c93344099b70e14e2837a62be', 'request_id': 'req-010eacb1-7af7-4068-889d-9b8ddfd25c38'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:58:01.631483 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-010eacb1-7af7-4068-889d-9b8ddfd25c38 admin admin] The token cache is being invalidate because project cf7a2c8c93344099b70e14e2837a62be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:58:01.631696 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-010eacb1-7af7-4068-889d-9b8ddfd25c38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:58:01.632329 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-010eacb1-7af7-4068-889d-9b8ddfd25c38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:58:01.633875 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1364/2712] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:58:01 2022] DELETE /identity/v3/projects/cf7a2c8c93344099b70e14e2837a62be => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:58:01.638697 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b47ed-8fe5-4ac1-9da4-885b4cf09499 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:58:01.639016 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b47ed-8fe5-4ac1-9da4-885b4cf09499 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:58:01.639297 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b47ed-8fe5-4ac1-9da4-885b4cf09499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:58:01.656897 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-332b47ed-8fe5-4ac1-9da4-885b4cf09499 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:58:01.658151 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-332b47ed-8fe5-4ac1-9da4-885b4cf09499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:58:01.692066 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:58:01.718147 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1349/2713] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:58:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:58:01.725951 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecec6a1-27ad-4cbd-9d26-23d7afc116a2 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:58:01.726239 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecec6a1-27ad-4cbd-9d26-23d7afc116a2 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:58:01.726517 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecec6a1-27ad-4cbd-9d26-23d7afc116a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:58:01.744803 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-9ecec6a1-27ad-4cbd-9d26-23d7afc116a2 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:58:01.746006 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-9ecec6a1-27ad-4cbd-9d26-23d7afc116a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:58:01.753764 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1365/2714] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:58:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:58:01.757953 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332b47ed-8fe5-4ac1-9da4-885b4cf09499 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:58:01.767555 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332b47ed-8fe5-4ac1-9da4-885b4cf09499 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:58:01.772774 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba14eed0-6467-4f21-ac51-bd49e3d77b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:58:01.773429 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba14eed0-6467-4f21-ac51-bd49e3d77b41 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:58:01.773807 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba14eed0-6467-4f21-ac51-bd49e3d77b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:58:01.774101 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba14eed0-6467-4f21-ac51-bd49e3d77b41 admin admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:58:01.774907 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba14eed0-6467-4f21-ac51-bd49e3d77b41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:58:01.775219 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba14eed0-6467-4f21-ac51-bd49e3d77b41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:58:01.776835 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba14eed0-6467-4f21-ac51-bd49e3d77b41 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:58:01.789512 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1350/2715] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 2 19:58:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:58:01.793848 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecec6a1-27ad-4cbd-9d26-23d7afc116a2 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:58:01.803001 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecec6a1-27ad-4cbd-9d26-23d7afc116a2 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:58:01.807655 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe584196-69d6-43c6-9e5e-522784426cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:58:01.808410 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe584196-69d6-43c6-9e5e-522784426cb1 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:58:01.808699 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe584196-69d6-43c6-9e5e-522784426cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:58:01.808969 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe584196-69d6-43c6-9e5e-522784426cb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:58:01.809651 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe584196-69d6-43c6-9e5e-522784426cb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:58:01.811572 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe584196-69d6-43c6-9e5e-522784426cb1 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:58:01.943213 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1366/2716] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:58:01 2022] POST /identity/v3/projects => generated 399 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:58:01.947234 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba14eed0-6467-4f21-ac51-bd49e3d77b41 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:58:01.957119 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba14eed0-6467-4f21-ac51-bd49e3d77b41 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:58:02.022077 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75003624-6533-4b04-a0d1-bd014b9aafab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:58:02.022775 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75003624-6533-4b04-a0d1-bd014b9aafab admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:58:02.023079 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75003624-6533-4b04-a0d1-bd014b9aafab admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:58:02.023359 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75003624-6533-4b04-a0d1-bd014b9aafab admin admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:58:02.024073 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75003624-6533-4b04-a0d1-bd014b9aafab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:58:02.026010 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75003624-6533-4b04-a0d1-bd014b9aafab admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:58:02.107602 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1351/2717] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:58:01 2022] POST /identity/v3/users => generated 300 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:58:02.111133 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe584196-69d6-43c6-9e5e-522784426cb1 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:58:02.125575 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe584196-69d6-43c6-9e5e-522784426cb1 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:58:02.130398 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ec35c5-3941-4b18-ae2c-9c64c6321fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:58:02.131104 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec35c5-3941-4b18-ae2c-9c64c6321fee admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:58:02.131393 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec35c5-3941-4b18-ae2c-9c64c6321fee admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:58:02.131696 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec35c5-3941-4b18-ae2c-9c64c6321fee admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:58:02.132428 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ec35c5-3941-4b18-ae2c-9c64c6321fee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:58:02.134083 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ec35c5-3941-4b18-ae2c-9c64c6321fee admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:58:02.144483 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1367/2718] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:58:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:58:02.148501 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75003624-6533-4b04-a0d1-bd014b9aafab admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:58:02.158399 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75003624-6533-4b04-a0d1-bd014b9aafab admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:58:02.168549 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c873f774-5358-4bce-8948-83268409b572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:58:02.168549 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c873f774-5358-4bce-8948-83268409b572 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:58:02.168549 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c873f774-5358-4bce-8948-83268409b572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:58:02.168549 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c873f774-5358-4bce-8948-83268409b572 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:58:02.168914 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c873f774-5358-4bce-8948-83268409b572 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:58:02.168914 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c873f774-5358-4bce-8948-83268409b572 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:58:02.178009 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1352/2719] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:58:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 02 19:58:02.181890 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ec35c5-3941-4b18-ae2c-9c64c6321fee admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:58:02.191601 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ec35c5-3941-4b18-ae2c-9c64c6321fee admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:58:02.196306 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b61fd1-f309-49e6-b3b9-c86d18049fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:58:02.197179 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b61fd1-f309-49e6-b3b9-c86d18049fde admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:58:02.197457 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b61fd1-f309-49e6-b3b9-c86d18049fde admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:58:02.197734 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b61fd1-f309-49e6-b3b9-c86d18049fde admin admin] PATH_INFO: `/v3/projects/baa86289135548eb9c103240a89c2cf1/users/d2ecf690344841a0b034289628d03a6a/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:58:02.199982 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:58:02.212751 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b61fd1-f309-49e6-b3b9-c86d18049fde admin admin] RBAC: Authorizing `identity:create_grant(project_id=baa86289135548eb9c103240a89c2cf1, user_id=d2ecf690344841a0b034289628d03a6a, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:58:02.215488 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b61fd1-f309-49e6-b3b9-c86d18049fde admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:58:02.318697 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1368/2720] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:58:02 2022] PUT /identity/v3/projects/baa86289135548eb9c103240a89c2cf1/users/d2ecf690344841a0b034289628d03a6a/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:58:02.334242 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c91740f-6e3c-41f4-9e90-d9d93effffc1 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:58:02.334559 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c91740f-6e3c-41f4-9e90-d9d93effffc1 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:58:02.334872 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c91740f-6e3c-41f4-9e90-d9d93effffc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:58:02.364068 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-9c91740f-6e3c-41f4-9e90-d9d93effffc1 None None] MFA Rules not processed for user `d2ecf690344841a0b034289628d03a6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:58:02.365939 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-9c91740f-6e3c-41f4-9e90-d9d93effffc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:58:02.428785 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1353/2721] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:58:02 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:58:02.440212 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b61fd1-f309-49e6-b3b9-c86d18049fde admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:58:02.442061 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-89b61fd1-f309-49e6-b3b9-c86d18049fde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:58:02.498285 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b61fd1-f309-49e6-b3b9-c86d18049fde admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:58:02.501887 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dcfa79-2967-4f83-b4a2-a3554b70b521 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:58:02.502525 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dcfa79-2967-4f83-b4a2-a3554b70b521 service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:58:02.502836 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dcfa79-2967-4f83-b4a2-a3554b70b521 service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:58:02.503114 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dcfa79-2967-4f83-b4a2-a3554b70b521 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:58:02.512536 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dcfa79-2967-4f83-b4a2-a3554b70b521 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:58:02.513987 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dcfa79-2967-4f83-b4a2-a3554b70b521 service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:58:02.524072 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1369/2722] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:58:02 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:58:11.525140 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c91740f-6e3c-41f4-9e90-d9d93effffc1 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:58:11.563309 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:58:11.585569 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c91740f-6e3c-41f4-9e90-d9d93effffc1 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:58:11.590123 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617bd8eb-2319-4c74-9a09-bd8ce445b4f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:58:11.591020 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617bd8eb-2319-4c74-9a09-bd8ce445b4f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:58:11.591304 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617bd8eb-2319-4c74-9a09-bd8ce445b4f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:58:11.591577 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617bd8eb-2319-4c74-9a09-bd8ce445b4f2 admin admin] PATH_INFO: `/v3/users/d2ecf690344841a0b034289628d03a6a` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:58:11.592744 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617bd8eb-2319-4c74-9a09-bd8ce445b4f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2ecf690344841a0b034289628d03a6a)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:58:11.594660 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617bd8eb-2319-4c74-9a09-bd8ce445b4f2 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:58:11.776218 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-617bd8eb-2319-4c74-9a09-bd8ce445b4f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2ecf690344841a0b034289628d03a6a'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:58:11.783280 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-617bd8eb-2319-4c74-9a09-bd8ce445b4f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2ecf690344841a0b034289628d03a6a'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:58:11.784013 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-617bd8eb-2319-4c74-9a09-bd8ce445b4f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2ecf690344841a0b034289628d03a6a'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:58:11.871237 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-617bd8eb-2319-4c74-9a09-bd8ce445b4f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2ecf690344841a0b034289628d03a6a'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:58:11.871813 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-617bd8eb-2319-4c74-9a09-bd8ce445b4f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2ecf690344841a0b034289628d03a6a'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:58:11.890411 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1354/2723] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:58:11 2022] DELETE /identity/v3/users/d2ecf690344841a0b034289628d03a6a => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:58:11.895214 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef516e-ca4c-4280-ad78-d78a42841eb6 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:58:11.895692 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef516e-ca4c-4280-ad78-d78a42841eb6 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:58:11.895819 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef516e-ca4c-4280-ad78-d78a42841eb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:58:11.915097 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-5bef516e-ca4c-4280-ad78-d78a42841eb6 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:58:11.916316 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-5bef516e-ca4c-4280-ad78-d78a42841eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:58:11.975452 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1370/2724] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:58:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:58:11.985742 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617bd8eb-2319-4c74-9a09-bd8ce445b4f2 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:58:11.987374 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-617bd8eb-2319-4c74-9a09-bd8ce445b4f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:58:12.045260 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617bd8eb-2319-4c74-9a09-bd8ce445b4f2 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:58:12.048995 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e184543-79a6-45ce-99b3-5939cf081d45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:58:12.049652 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e184543-79a6-45ce-99b3-5939cf081d45 service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:58:12.049941 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e184543-79a6-45ce-99b3-5939cf081d45 service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:58:12.050224 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e184543-79a6-45ce-99b3-5939cf081d45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:58:12.059449 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e184543-79a6-45ce-99b3-5939cf081d45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:58:12.060879 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e184543-79a6-45ce-99b3-5939cf081d45 service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:58:12.070589 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1355/2725] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:58:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:58:12.461534 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef516e-ca4c-4280-ad78-d78a42841eb6 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:58:12.463085 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-5bef516e-ca4c-4280-ad78-d78a42841eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:58:12.475023 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef516e-ca4c-4280-ad78-d78a42841eb6 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:58:12.479679 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aceaebb-294e-41c4-99f8-68c93a4018c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:58:12.480551 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aceaebb-294e-41c4-99f8-68c93a4018c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:58:12.480835 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aceaebb-294e-41c4-99f8-68c93a4018c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:58:12.481112 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aceaebb-294e-41c4-99f8-68c93a4018c3 admin admin] PATH_INFO: `/v3/projects/baa86289135548eb9c103240a89c2cf1` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:58:12.482333 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aceaebb-294e-41c4-99f8-68c93a4018c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=baa86289135548eb9c103240a89c2cf1)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:58:12.484318 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aceaebb-294e-41c4-99f8-68c93a4018c3 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:58:12.584408 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-6aceaebb-294e-41c4-99f8-68c93a4018c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'baa86289135548eb9c103240a89c2cf1', 'request_id': 'req-6aceaebb-294e-41c4-99f8-68c93a4018c3'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:58:12.665443 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-6aceaebb-294e-41c4-99f8-68c93a4018c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'baa86289135548eb9c103240a89c2cf1', 'request_id': 'req-6aceaebb-294e-41c4-99f8-68c93a4018c3'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:58:12.671135 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-6aceaebb-294e-41c4-99f8-68c93a4018c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'baa86289135548eb9c103240a89c2cf1', 'request_id': 'req-6aceaebb-294e-41c4-99f8-68c93a4018c3'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:58:12.682068 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-6aceaebb-294e-41c4-99f8-68c93a4018c3 admin admin] The token cache is being invalidate because project baa86289135548eb9c103240a89c2cf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:58:12.682359 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-6aceaebb-294e-41c4-99f8-68c93a4018c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:58:12.683013 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-6aceaebb-294e-41c4-99f8-68c93a4018c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:58:12.684555 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1371/2726] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:58:12 2022] DELETE /identity/v3/projects/baa86289135548eb9c103240a89c2cf1 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:58:14.279304 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e184543-79a6-45ce-99b3-5939cf081d45 service neutron] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:58:14.280826 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-8e184543-79a6-45ce-99b3-5939cf081d45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:58:14.292816 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e184543-79a6-45ce-99b3-5939cf081d45 service neutron] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:58:14.340195 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c31ca9e-014d-4ff4-b439-d41143198cac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:58:14.340838 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c31ca9e-014d-4ff4-b439-d41143198cac service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:58:14.341123 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c31ca9e-014d-4ff4-b439-d41143198cac service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:58:14.341403 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c31ca9e-014d-4ff4-b439-d41143198cac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:58:14.343171 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-4c31ca9e-014d-4ff4-b439-d41143198cac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:58:14.352854 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c31ca9e-014d-4ff4-b439-d41143198cac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:58:14.354279 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c31ca9e-014d-4ff4-b439-d41143198cac service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:58:14.358322 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:58:14.362113 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1356/2727] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:58:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:58:22.679072 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:59:20.896377 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aceaebb-294e-41c4-99f8-68c93a4018c3 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:59:20.897930 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-6aceaebb-294e-41c4-99f8-68c93a4018c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:59:20.912938 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aceaebb-294e-41c4-99f8-68c93a4018c3 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:59:20.916755 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253bd5b2-dc8d-4046-ae80-d3a1c686de36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:59:20.917401 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253bd5b2-dc8d-4046-ae80-d3a1c686de36 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:59:20.917690 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253bd5b2-dc8d-4046-ae80-d3a1c686de36 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:59:20.917968 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253bd5b2-dc8d-4046-ae80-d3a1c686de36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:59:20.919757 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-253bd5b2-dc8d-4046-ae80-d3a1c686de36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:59:20.983453 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253bd5b2-dc8d-4046-ae80-d3a1c686de36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:59:20.985005 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253bd5b2-dc8d-4046-ae80-d3a1c686de36 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:59:20.994656 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1372/2728] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:59:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:59:24.547293 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c31ca9e-014d-4ff4-b439-d41143198cac service neutron] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:59:24.548946 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-4c31ca9e-014d-4ff4-b439-d41143198cac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:59:24.565591 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c31ca9e-014d-4ff4-b439-d41143198cac service neutron] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:59:24.570074 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f6d26f-a49f-4647-8916-b807d5e4ef77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:59:24.570983 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f6d26f-a49f-4647-8916-b807d5e4ef77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:59:24.571268 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f6d26f-a49f-4647-8916-b807d5e4ef77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:59:24.571547 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f6d26f-a49f-4647-8916-b807d5e4ef77 admin admin] PATH_INFO: `/v3/users/df62e9ab4d044dcaab00c5d76416e3e1` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:59:24.572870 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f6d26f-a49f-4647-8916-b807d5e4ef77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df62e9ab4d044dcaab00c5d76416e3e1)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:59:24.574783 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f6d26f-a49f-4647-8916-b807d5e4ef77 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:59:24.769773 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-37f6d26f-a49f-4647-8916-b807d5e4ef77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df62e9ab4d044dcaab00c5d76416e3e1'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:59:24.777087 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-37f6d26f-a49f-4647-8916-b807d5e4ef77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df62e9ab4d044dcaab00c5d76416e3e1'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:59:24.777799 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-37f6d26f-a49f-4647-8916-b807d5e4ef77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df62e9ab4d044dcaab00c5d76416e3e1'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:59:24.860359 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-37f6d26f-a49f-4647-8916-b807d5e4ef77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df62e9ab4d044dcaab00c5d76416e3e1'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:59:24.860996 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-37f6d26f-a49f-4647-8916-b807d5e4ef77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df62e9ab4d044dcaab00c5d76416e3e1'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:59:24.879389 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1357/2729] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:59:24 2022] DELETE /identity/v3/users/df62e9ab4d044dcaab00c5d76416e3e1 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:59:25.132808 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253bd5b2-dc8d-4046-ae80-d3a1c686de36 service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:59:25.141136 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253bd5b2-dc8d-4046-ae80-d3a1c686de36 service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:59:25.205386 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42a6c75-6480-4450-90f4-0c0d087e9bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:59:25.206259 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a6c75-6480-4450-90f4-0c0d087e9bf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:59:25.206657 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a6c75-6480-4450-90f4-0c0d087e9bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:59:25.206975 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a6c75-6480-4450-90f4-0c0d087e9bf4 admin admin] PATH_INFO: `/v3/projects/626d49f321144516b095148aa0f632dc` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:59:25.208168 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42a6c75-6480-4450-90f4-0c0d087e9bf4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=626d49f321144516b095148aa0f632dc)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:59:25.210089 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42a6c75-6480-4450-90f4-0c0d087e9bf4 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:59:25.311856 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-b42a6c75-6480-4450-90f4-0c0d087e9bf4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '626d49f321144516b095148aa0f632dc', 'request_id': 'req-b42a6c75-6480-4450-90f4-0c0d087e9bf4'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:59:25.391823 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-b42a6c75-6480-4450-90f4-0c0d087e9bf4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '626d49f321144516b095148aa0f632dc', 'request_id': 'req-b42a6c75-6480-4450-90f4-0c0d087e9bf4'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:59:25.397886 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-b42a6c75-6480-4450-90f4-0c0d087e9bf4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '626d49f321144516b095148aa0f632dc', 'request_id': 'req-b42a6c75-6480-4450-90f4-0c0d087e9bf4'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:59:25.409174 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-b42a6c75-6480-4450-90f4-0c0d087e9bf4 admin admin] The token cache is being invalidate because project 626d49f321144516b095148aa0f632dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:59:25.409470 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-b42a6c75-6480-4450-90f4-0c0d087e9bf4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:59:25.410143 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-b42a6c75-6480-4450-90f4-0c0d087e9bf4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:59:25.411835 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1373/2730] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:59:25 2022] DELETE /identity/v3/projects/626d49f321144516b095148aa0f632dc => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:59:25.417543 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c5521-be0c-4728-b864-d5839d5430e9 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:59:25.417841 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c5521-be0c-4728-b864-d5839d5430e9 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:59:25.418129 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c5521-be0c-4728-b864-d5839d5430e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:59:25.437454 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-5d6c5521-be0c-4728-b864-d5839d5430e9 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:59:25.439408 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-5d6c5521-be0c-4728-b864-d5839d5430e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:59:25.500463 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1358/2731] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:59:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:59:25.509100 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ea64d7-4667-4eff-af32-bbc3ab4460a4 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:59:25.509394 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ea64d7-4667-4eff-af32-bbc3ab4460a4 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:59:25.509676 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ea64d7-4667-4eff-af32-bbc3ab4460a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:59:25.527443 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-29ea64d7-4667-4eff-af32-bbc3ab4460a4 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:59:25.528660 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-29ea64d7-4667-4eff-af32-bbc3ab4460a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:59:25.536472 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1374/2732] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:59:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:59:25.540441 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6c5521-be0c-4728-b864-d5839d5430e9 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:59:25.550490 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6c5521-be0c-4728-b864-d5839d5430e9 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:59:25.555104 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0dd177-787d-429c-abd6-0066ae5c5a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:59:25.555753 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0dd177-787d-429c-abd6-0066ae5c5a13 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:59:25.556042 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0dd177-787d-429c-abd6-0066ae5c5a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:59:25.556326 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0dd177-787d-429c-abd6-0066ae5c5a13 admin admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:59:25.557055 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0dd177-787d-429c-abd6-0066ae5c5a13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:59:25.557367 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0dd177-787d-429c-abd6-0066ae5c5a13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:59:25.558989 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0dd177-787d-429c-abd6-0066ae5c5a13 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:59:25.571114 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1359/2733] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 2 19:59:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:59:25.575248 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ea64d7-4667-4eff-af32-bbc3ab4460a4 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:59:25.584438 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ea64d7-4667-4eff-af32-bbc3ab4460a4 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:59:25.589068 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72ee755-f4a7-4f2d-b7bf-a6abd9c27a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:59:25.589715 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72ee755-f4a7-4f2d-b7bf-a6abd9c27a60 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:59:25.590005 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72ee755-f4a7-4f2d-b7bf-a6abd9c27a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:59:25.590290 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72ee755-f4a7-4f2d-b7bf-a6abd9c27a60 admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:59:25.591049 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72ee755-f4a7-4f2d-b7bf-a6abd9c27a60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:59:25.592948 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72ee755-f4a7-4f2d-b7bf-a6abd9c27a60 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:59:25.607435 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:59:25.684425 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1375/2734] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:59:25 2022] POST /identity/v3/projects => generated 405 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:59:25.688776 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0dd177-787d-429c-abd6-0066ae5c5a13 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:59:25.698836 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0dd177-787d-429c-abd6-0066ae5c5a13 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:59:25.754608 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7097e9-b592-4b55-ae60-597a5da22343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:59:25.755382 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7097e9-b592-4b55-ae60-597a5da22343 admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:59:25.755666 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7097e9-b592-4b55-ae60-597a5da22343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:59:25.755938 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7097e9-b592-4b55-ae60-597a5da22343 admin admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:59:25.756620 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7097e9-b592-4b55-ae60-597a5da22343 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:59:25.758524 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7097e9-b592-4b55-ae60-597a5da22343 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:59:25.765182 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:59:25.825945 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1360/2735] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:59:25 2022] POST /identity/v3/users => generated 303 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:59:25.830194 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72ee755-f4a7-4f2d-b7bf-a6abd9c27a60 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:59:25.840641 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72ee755-f4a7-4f2d-b7bf-a6abd9c27a60 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:59:25.845744 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43672653-b984-4b78-a297-ac97457225cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:59:25.846406 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43672653-b984-4b78-a297-ac97457225cc admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:59:25.846692 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43672653-b984-4b78-a297-ac97457225cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:59:25.847006 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43672653-b984-4b78-a297-ac97457225cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:59:25.847812 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43672653-b984-4b78-a297-ac97457225cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:59:25.849335 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43672653-b984-4b78-a297-ac97457225cc admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:59:25.860502 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1376/2736] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:59:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:59:25.864934 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7097e9-b592-4b55-ae60-597a5da22343 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:59:25.874712 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7097e9-b592-4b55-ae60-597a5da22343 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:59:25.879366 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a00a6d3-6d09-4e4a-8923-c5128f1ff6db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:59:25.881852 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00a6d3-6d09-4e4a-8923-c5128f1ff6db admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:59:25.881852 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00a6d3-6d09-4e4a-8923-c5128f1ff6db admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:59:25.881852 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00a6d3-6d09-4e4a-8923-c5128f1ff6db admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:59:25.881852 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a00a6d3-6d09-4e4a-8923-c5128f1ff6db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:59:25.882882 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a00a6d3-6d09-4e4a-8923-c5128f1ff6db admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:59:25.893426 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1361/2737] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:59:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:59:25.897297 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43672653-b984-4b78-a297-ac97457225cc admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:59:25.907076 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43672653-b984-4b78-a297-ac97457225cc admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:59:25.912202 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab15f4c-75c6-45ec-9333-f33cbf88d6aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:59:25.913079 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab15f4c-75c6-45ec-9333-f33cbf88d6aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:59:25.913367 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab15f4c-75c6-45ec-9333-f33cbf88d6aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:59:25.913648 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab15f4c-75c6-45ec-9333-f33cbf88d6aa admin admin] PATH_INFO: `/v3/projects/7749c0dc75a242b5a28964a5b63d1d2c/users/bf1b7ab29de94cd290b192c73f7edf9b/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:59:25.928647 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab15f4c-75c6-45ec-9333-f33cbf88d6aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=7749c0dc75a242b5a28964a5b63d1d2c, user_id=bf1b7ab29de94cd290b192c73f7edf9b, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:59:25.931333 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab15f4c-75c6-45ec-9333-f33cbf88d6aa admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:59:26.013045 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1377/2738] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:59:25 2022] PUT /identity/v3/projects/7749c0dc75a242b5a28964a5b63d1d2c/users/bf1b7ab29de94cd290b192c73f7edf9b/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:59:26.023954 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec6fc5f-2ca7-417b-a527-d558ab925dfa None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:59:26.024276 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec6fc5f-2ca7-417b-a527-d558ab925dfa None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:59:26.024610 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec6fc5f-2ca7-417b-a527-d558ab925dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:59:26.052395 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-bec6fc5f-2ca7-417b-a527-d558ab925dfa None None] MFA Rules not processed for user `bf1b7ab29de94cd290b192c73f7edf9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:59:26.053599 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-bec6fc5f-2ca7-417b-a527-d558ab925dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:59:26.251000 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1362/2739] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:59:26 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:59:26.255991 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab15f4c-75c6-45ec-9333-f33cbf88d6aa admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:59:26.317776 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab15f4c-75c6-45ec-9333-f33cbf88d6aa admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:59:26.322248 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0ef9d4-e748-4361-aca3-a1fb02011a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:59:26.322948 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0ef9d4-e748-4361-aca3-a1fb02011a17 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:59:26.323221 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0ef9d4-e748-4361-aca3-a1fb02011a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:59:26.323500 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0ef9d4-e748-4361-aca3-a1fb02011a17 admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:59:26.324185 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0ef9d4-e748-4361-aca3-a1fb02011a17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:59:26.326113 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0ef9d4-e748-4361-aca3-a1fb02011a17 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:59:26.403533 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1378/2740] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:59:26 2022] POST /identity/v3/projects => generated 405 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:59:26.407487 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec6fc5f-2ca7-417b-a527-d558ab925dfa None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:59:26.422589 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec6fc5f-2ca7-417b-a527-d558ab925dfa None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:59:26.472859 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef0a734-49e5-479b-a579-b6caac19d4d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:59:26.473533 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef0a734-49e5-479b-a579-b6caac19d4d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:59:26.473812 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef0a734-49e5-479b-a579-b6caac19d4d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:59:26.474145 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef0a734-49e5-479b-a579-b6caac19d4d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:59:26.474865 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef0a734-49e5-479b-a579-b6caac19d4d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:59:26.476743 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef0a734-49e5-479b-a579-b6caac19d4d7 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:59:26.563506 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1363/2741] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:59:26 2022] POST /identity/v3/users => generated 309 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:59:26.568081 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0ef9d4-e748-4361-aca3-a1fb02011a17 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:59:26.578641 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0ef9d4-e748-4361-aca3-a1fb02011a17 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:59:26.583893 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e67807-bf65-4936-987e-0ca0d6beea76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:59:26.584560 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e67807-bf65-4936-987e-0ca0d6beea76 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:59:26.584881 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e67807-bf65-4936-987e-0ca0d6beea76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:59:26.585167 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e67807-bf65-4936-987e-0ca0d6beea76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:59:26.585896 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e67807-bf65-4936-987e-0ca0d6beea76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:59:26.587537 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e67807-bf65-4936-987e-0ca0d6beea76 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:59:26.598554 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1379/2742] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:59:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:59:26.602728 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef0a734-49e5-479b-a579-b6caac19d4d7 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:59:26.612881 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef0a734-49e5-479b-a579-b6caac19d4d7 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:59:26.617497 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c12cff-2efd-4744-92b0-733c687cc8f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:59:26.618370 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c12cff-2efd-4744-92b0-733c687cc8f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:59:26.618658 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c12cff-2efd-4744-92b0-733c687cc8f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:59:26.618967 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c12cff-2efd-4744-92b0-733c687cc8f8 admin admin] PATH_INFO: `/v3/projects/c7201bd248ac45fd993164b0658dbe13/users/28f3400244204e9d958ee7bacf4e9222/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:59:26.633943 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c12cff-2efd-4744-92b0-733c687cc8f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7201bd248ac45fd993164b0658dbe13, user_id=28f3400244204e9d958ee7bacf4e9222, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:59:26.636680 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c12cff-2efd-4744-92b0-733c687cc8f8 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:59:26.705886 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1364/2743] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:59:26 2022] PUT /identity/v3/projects/c7201bd248ac45fd993164b0658dbe13/users/28f3400244204e9d958ee7bacf4e9222/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:59:26.709774 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e67807-bf65-4936-987e-0ca0d6beea76 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:59:26.719699 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e67807-bf65-4936-987e-0ca0d6beea76 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:59:26.775934 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67951be9-af13-44a1-87d0-28f384e22f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:59:26.776606 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67951be9-af13-44a1-87d0-28f384e22f6a admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:59:26.776894 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67951be9-af13-44a1-87d0-28f384e22f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:59:26.777178 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67951be9-af13-44a1-87d0-28f384e22f6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:59:26.777909 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67951be9-af13-44a1-87d0-28f384e22f6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:59:26.779551 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67951be9-af13-44a1-87d0-28f384e22f6a admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:59:26.789541 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1380/2744] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:59:26 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:59:26.793591 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c12cff-2efd-4744-92b0-733c687cc8f8 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:59:26.803972 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c12cff-2efd-4744-92b0-733c687cc8f8 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:59:26.808443 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904d35e9-35e8-4ce0-8843-afbdce2598d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:59:26.809313 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904d35e9-35e8-4ce0-8843-afbdce2598d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:59:26.809594 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904d35e9-35e8-4ce0-8843-afbdce2598d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:59:26.809869 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904d35e9-35e8-4ce0-8843-afbdce2598d4 admin admin] PATH_INFO: `/v3/projects/c7201bd248ac45fd993164b0658dbe13/users/28f3400244204e9d958ee7bacf4e9222/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:59:26.811795 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904d35e9-35e8-4ce0-8843-afbdce2598d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7201bd248ac45fd993164b0658dbe13, user_id=28f3400244204e9d958ee7bacf4e9222, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:59:26.814365 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904d35e9-35e8-4ce0-8843-afbdce2598d4 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:59:26.824854 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1365/2745] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:59:26 2022] PUT /identity/v3/projects/c7201bd248ac45fd993164b0658dbe13/users/28f3400244204e9d958ee7bacf4e9222/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 02 19:59:26.834804 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3f38bf-7b9b-4d44-9bd3-7015ffc5ad6f None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:59:26.835105 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3f38bf-7b9b-4d44-9bd3-7015ffc5ad6f None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:59:26.835382 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3f38bf-7b9b-4d44-9bd3-7015ffc5ad6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:59:26.860882 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-0b3f38bf-7b9b-4d44-9bd3-7015ffc5ad6f None None] MFA Rules not processed for user `28f3400244204e9d958ee7bacf4e9222`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:59:26.862587 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-0b3f38bf-7b9b-4d44-9bd3-7015ffc5ad6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:59:26.880688 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:59:26.930178 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1381/2746] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:59:26 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:59:26.941146 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904d35e9-35e8-4ce0-8843-afbdce2598d4 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:59:26.942812 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-904d35e9-35e8-4ce0-8843-afbdce2598d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:59:26.964137 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:59:26.998432 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904d35e9-35e8-4ce0-8843-afbdce2598d4 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:59:27.003117 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f5ab53-b879-4721-97a5-ad4d09b307ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:59:27.003764 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f5ab53-b879-4721-97a5-ad4d09b307ea service cinder] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:59:27.004057 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f5ab53-b879-4721-97a5-ad4d09b307ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:59:27.004339 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f5ab53-b879-4721-97a5-ad4d09b307ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:59:27.013336 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f5ab53-b879-4721-97a5-ad4d09b307ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:59:27.014779 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f5ab53-b879-4721-97a5-ad4d09b307ea service cinder] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:59:27.024007 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1366/2747] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:59:26 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:59:27.150898 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3f38bf-7b9b-4d44-9bd3-7015ffc5ad6f None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:59:27.211886 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3f38bf-7b9b-4d44-9bd3-7015ffc5ad6f None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:59:27.216418 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da244cdf-0537-4050-abe3-71efe79fda95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:59:27.217233 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da244cdf-0537-4050-abe3-71efe79fda95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:59:27.217520 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da244cdf-0537-4050-abe3-71efe79fda95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:59:27.217799 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da244cdf-0537-4050-abe3-71efe79fda95 admin admin] PATH_INFO: `/v3/users/bf1b7ab29de94cd290b192c73f7edf9b` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:59:27.219036 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da244cdf-0537-4050-abe3-71efe79fda95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf1b7ab29de94cd290b192c73f7edf9b)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:59:27.220969 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da244cdf-0537-4050-abe3-71efe79fda95 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:59:27.392761 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-da244cdf-0537-4050-abe3-71efe79fda95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf1b7ab29de94cd290b192c73f7edf9b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:59:27.400224 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-da244cdf-0537-4050-abe3-71efe79fda95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf1b7ab29de94cd290b192c73f7edf9b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:59:27.460131 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-da244cdf-0537-4050-abe3-71efe79fda95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf1b7ab29de94cd290b192c73f7edf9b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:59:27.460750 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-da244cdf-0537-4050-abe3-71efe79fda95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf1b7ab29de94cd290b192c73f7edf9b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:59:27.461363 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-da244cdf-0537-4050-abe3-71efe79fda95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf1b7ab29de94cd290b192c73f7edf9b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:59:27.479747 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1382/2748] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:59:27 2022] DELETE /identity/v3/users/bf1b7ab29de94cd290b192c73f7edf9b => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:59:27.484726 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2af98c-e4fa-432c-9434-2eef1f4c8159 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:59:27.485016 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2af98c-e4fa-432c-9434-2eef1f4c8159 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:59:27.485296 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2af98c-e4fa-432c-9434-2eef1f4c8159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:59:27.502302 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-ea2af98c-e4fa-432c-9434-2eef1f4c8159 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:59:27.503543 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-ea2af98c-e4fa-432c-9434-2eef1f4c8159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:59:27.559970 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1367/2749] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:59:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:59:27.569545 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da244cdf-0537-4050-abe3-71efe79fda95 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:59:27.571007 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-da244cdf-0537-4050-abe3-71efe79fda95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:59:27.612619 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:59:27.629207 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da244cdf-0537-4050-abe3-71efe79fda95 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:59:27.632832 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbad5513-b5a2-45dd-b6d6-64eccb507c50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:59:27.633453 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbad5513-b5a2-45dd-b6d6-64eccb507c50 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:59:27.633736 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbad5513-b5a2-45dd-b6d6-64eccb507c50 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:59:27.634012 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbad5513-b5a2-45dd-b6d6-64eccb507c50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:59:27.643629 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbad5513-b5a2-45dd-b6d6-64eccb507c50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:59:27.645057 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbad5513-b5a2-45dd-b6d6-64eccb507c50 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:59:27.654287 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1383/2750] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:59:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 02 19:59:28.119697 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2af98c-e4fa-432c-9434-2eef1f4c8159 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:59:28.121208 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-ea2af98c-e4fa-432c-9434-2eef1f4c8159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:59:28.137231 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2af98c-e4fa-432c-9434-2eef1f4c8159 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:59:28.142364 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b4fed6-720f-4658-93e0-60cf386bb4be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:59:28.143918 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4fed6-720f-4658-93e0-60cf386bb4be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:59:28.144198 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4fed6-720f-4658-93e0-60cf386bb4be admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:59:28.144473 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4fed6-720f-4658-93e0-60cf386bb4be admin admin] PATH_INFO: `/v3/projects/7749c0dc75a242b5a28964a5b63d1d2c` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:59:28.145659 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b4fed6-720f-4658-93e0-60cf386bb4be admin admin] RBAC: Authorizing `identity:delete_project(project_id=7749c0dc75a242b5a28964a5b63d1d2c)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:59:28.147655 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b4fed6-720f-4658-93e0-60cf386bb4be admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:59:28.256063 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-86b4fed6-720f-4658-93e0-60cf386bb4be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7749c0dc75a242b5a28964a5b63d1d2c', 'request_id': 'req-86b4fed6-720f-4658-93e0-60cf386bb4be'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:59:28.327211 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-86b4fed6-720f-4658-93e0-60cf386bb4be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7749c0dc75a242b5a28964a5b63d1d2c', 'request_id': 'req-86b4fed6-720f-4658-93e0-60cf386bb4be'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:59:28.332942 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-86b4fed6-720f-4658-93e0-60cf386bb4be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7749c0dc75a242b5a28964a5b63d1d2c', 'request_id': 'req-86b4fed6-720f-4658-93e0-60cf386bb4be'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:59:28.343940 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-86b4fed6-720f-4658-93e0-60cf386bb4be admin admin] The token cache is being invalidate because project 7749c0dc75a242b5a28964a5b63d1d2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:59:28.344252 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-86b4fed6-720f-4658-93e0-60cf386bb4be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:59:28.344879 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-86b4fed6-720f-4658-93e0-60cf386bb4be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:59:28.346400 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1368/2751] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:59:28 2022] DELETE /identity/v3/projects/7749c0dc75a242b5a28964a5b63d1d2c => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:59:28.350521 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbad5513-b5a2-45dd-b6d6-64eccb507c50 service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:59:28.360281 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbad5513-b5a2-45dd-b6d6-64eccb507c50 service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:59:28.416011 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7099f316-e678-4379-96f9-0897a43fb9d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:59:28.416801 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7099f316-e678-4379-96f9-0897a43fb9d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:59:28.417079 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7099f316-e678-4379-96f9-0897a43fb9d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:59:28.417352 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7099f316-e678-4379-96f9-0897a43fb9d1 admin admin] PATH_INFO: `/v3/users/28f3400244204e9d958ee7bacf4e9222` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:59:28.418515 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7099f316-e678-4379-96f9-0897a43fb9d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28f3400244204e9d958ee7bacf4e9222)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:59:28.420547 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7099f316-e678-4379-96f9-0897a43fb9d1 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:59:28.599243 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-7099f316-e678-4379-96f9-0897a43fb9d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28f3400244204e9d958ee7bacf4e9222'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:59:28.606157 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-7099f316-e678-4379-96f9-0897a43fb9d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28f3400244204e9d958ee7bacf4e9222'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:59:28.694012 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-7099f316-e678-4379-96f9-0897a43fb9d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28f3400244204e9d958ee7bacf4e9222'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:59:28.694669 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-7099f316-e678-4379-96f9-0897a43fb9d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28f3400244204e9d958ee7bacf4e9222'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:59:28.695421 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-7099f316-e678-4379-96f9-0897a43fb9d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28f3400244204e9d958ee7bacf4e9222'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:59:28.713192 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1384/2752] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 19:59:28 2022] DELETE /identity/v3/users/28f3400244204e9d958ee7bacf4e9222 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:59:29.119965 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b4fed6-720f-4658-93e0-60cf386bb4be admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:59:29.122482 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-86b4fed6-720f-4658-93e0-60cf386bb4be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:59:29.137637 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b4fed6-720f-4658-93e0-60cf386bb4be admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:59:29.184507 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:59:29.196844 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e303a0-0279-4045-b6d3-82e7fc99996c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:59:29.197706 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e303a0-0279-4045-b6d3-82e7fc99996c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:59:29.197994 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e303a0-0279-4045-b6d3-82e7fc99996c admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:59:29.198282 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e303a0-0279-4045-b6d3-82e7fc99996c admin admin] PATH_INFO: `/v3/projects/c7201bd248ac45fd993164b0658dbe13` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:59:29.199489 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e303a0-0279-4045-b6d3-82e7fc99996c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7201bd248ac45fd993164b0658dbe13)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:59:29.201475 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e303a0-0279-4045-b6d3-82e7fc99996c admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:59:29.299029 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-60e303a0-0279-4045-b6d3-82e7fc99996c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7201bd248ac45fd993164b0658dbe13', 'request_id': 'req-60e303a0-0279-4045-b6d3-82e7fc99996c'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:59:29.414053 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-60e303a0-0279-4045-b6d3-82e7fc99996c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7201bd248ac45fd993164b0658dbe13', 'request_id': 'req-60e303a0-0279-4045-b6d3-82e7fc99996c'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:59:29.419741 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-60e303a0-0279-4045-b6d3-82e7fc99996c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7201bd248ac45fd993164b0658dbe13', 'request_id': 'req-60e303a0-0279-4045-b6d3-82e7fc99996c'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:59:29.430582 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-60e303a0-0279-4045-b6d3-82e7fc99996c admin admin] The token cache is being invalidate because project c7201bd248ac45fd993164b0658dbe13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 19:59:29.430900 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-60e303a0-0279-4045-b6d3-82e7fc99996c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:59:29.431532 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-60e303a0-0279-4045-b6d3-82e7fc99996c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 19:59:29.433072 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1369/2753] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 19:59:29 2022] DELETE /identity/v3/projects/c7201bd248ac45fd993164b0658dbe13 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:59:29.438253 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cf1c17-0242-469e-b7c0-396d70a01f3f None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:59:29.438707 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cf1c17-0242-469e-b7c0-396d70a01f3f None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:59:29.438847 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cf1c17-0242-469e-b7c0-396d70a01f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:59:29.457276 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-83cf1c17-0242-469e-b7c0-396d70a01f3f None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:59:29.458414 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-83cf1c17-0242-469e-b7c0-396d70a01f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:59:29.506715 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:59:29.515535 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1385/2754] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:59:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:59:29.571943 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d573172e-5eed-47c2-86ff-e37cab6db992 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:59:29.572234 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d573172e-5eed-47c2-86ff-e37cab6db992 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:59:29.572552 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d573172e-5eed-47c2-86ff-e37cab6db992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:59:29.591292 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-d573172e-5eed-47c2-86ff-e37cab6db992 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:59:29.592485 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-d573172e-5eed-47c2-86ff-e37cab6db992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:59:29.600207 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1370/2755] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:59:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:59:29.604325 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cf1c17-0242-469e-b7c0-396d70a01f3f None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:59:29.614012 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cf1c17-0242-469e-b7c0-396d70a01f3f None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:59:29.618428 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f2f80d-0e52-4257-bb4a-a57a8046c211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:59:29.619109 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f2f80d-0e52-4257-bb4a-a57a8046c211 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:59:29.619391 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f2f80d-0e52-4257-bb4a-a57a8046c211 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:59:29.619668 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f2f80d-0e52-4257-bb4a-a57a8046c211 admin admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:59:29.620386 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f2f80d-0e52-4257-bb4a-a57a8046c211 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 19:59:29.620693 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f2f80d-0e52-4257-bb4a-a57a8046c211 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:59:29.622267 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f2f80d-0e52-4257-bb4a-a57a8046c211 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:59:29.634057 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1386/2756] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 2 19:59:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 19:59:29.638091 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d573172e-5eed-47c2-86ff-e37cab6db992 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:59:29.647660 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d573172e-5eed-47c2-86ff-e37cab6db992 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:59:29.652210 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50209066-b75b-485c-a14b-73499ae39d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:59:29.652850 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50209066-b75b-485c-a14b-73499ae39d0b admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:59:29.653136 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50209066-b75b-485c-a14b-73499ae39d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:59:29.653414 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50209066-b75b-485c-a14b-73499ae39d0b admin admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:59:29.654095 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50209066-b75b-485c-a14b-73499ae39d0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:59:29.656005 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50209066-b75b-485c-a14b-73499ae39d0b admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:59:29.735048 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1371/2757] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:59:29 2022] POST /identity/v3/projects => generated 371 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:59:29.739735 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f2f80d-0e52-4257-bb4a-a57a8046c211 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:59:29.749753 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f2f80d-0e52-4257-bb4a-a57a8046c211 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:59:29.805080 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a0833a-7e82-4504-a7a6-66059ca0c65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:59:29.805740 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a0833a-7e82-4504-a7a6-66059ca0c65f admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:59:29.806024 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a0833a-7e82-4504-a7a6-66059ca0c65f admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:59:29.806298 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a0833a-7e82-4504-a7a6-66059ca0c65f admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:59:29.807024 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a0833a-7e82-4504-a7a6-66059ca0c65f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:59:29.808929 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a0833a-7e82-4504-a7a6-66059ca0c65f admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:59:29.922272 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1387/2758] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:59:29 2022] POST /identity/v3/users => generated 286 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:59:29.926852 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50209066-b75b-485c-a14b-73499ae39d0b admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:59:29.937263 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50209066-b75b-485c-a14b-73499ae39d0b admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:59:29.942614 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6537c0f6-1923-4804-94b2-6bdcd452b85b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:59:29.943304 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6537c0f6-1923-4804-94b2-6bdcd452b85b admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:59:29.943592 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6537c0f6-1923-4804-94b2-6bdcd452b85b admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:59:29.943873 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6537c0f6-1923-4804-94b2-6bdcd452b85b admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:59:29.944605 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6537c0f6-1923-4804-94b2-6bdcd452b85b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:59:29.946281 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6537c0f6-1923-4804-94b2-6bdcd452b85b admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:59:29.957417 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1372/2759] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:59:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:59:29.961309 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a0833a-7e82-4504-a7a6-66059ca0c65f admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:59:29.970958 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a0833a-7e82-4504-a7a6-66059ca0c65f admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:59:29.975445 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d18f68d-c292-442f-9a1d-e6bba4d63ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:59:29.976087 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18f68d-c292-442f-9a1d-e6bba4d63ffb admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:59:29.976368 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18f68d-c292-442f-9a1d-e6bba4d63ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:59:29.976643 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18f68d-c292-442f-9a1d-e6bba4d63ffb admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:59:29.977362 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d18f68d-c292-442f-9a1d-e6bba4d63ffb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:59:29.979005 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d18f68d-c292-442f-9a1d-e6bba4d63ffb admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:59:29.989399 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1388/2760] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:59:29 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:59:29.993291 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6537c0f6-1923-4804-94b2-6bdcd452b85b admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:59:30.003691 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6537c0f6-1923-4804-94b2-6bdcd452b85b admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:59:30.008239 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23bd856-b826-4734-a03b-b8709bd8c684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:59:30.009103 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23bd856-b826-4734-a03b-b8709bd8c684 admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:59:30.009388 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23bd856-b826-4734-a03b-b8709bd8c684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:59:30.009666 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23bd856-b826-4734-a03b-b8709bd8c684 admin admin] PATH_INFO: `/v3/projects/abbbb1aa51c246eca467a32562ffd7ce/users/7634e72625254c12bc7427200135c55e/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:59:30.023868 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23bd856-b826-4734-a03b-b8709bd8c684 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abbbb1aa51c246eca467a32562ffd7ce, user_id=7634e72625254c12bc7427200135c55e, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:59:30.026455 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23bd856-b826-4734-a03b-b8709bd8c684 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:59:30.138715 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1373/2761] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:59:29 2022] PUT /identity/v3/projects/abbbb1aa51c246eca467a32562ffd7ce/users/7634e72625254c12bc7427200135c55e/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:59:30.149307 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701933b0-24ea-4b43-bd45-a64b38513b06 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:59:30.149607 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701933b0-24ea-4b43-bd45-a64b38513b06 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:59:30.149892 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701933b0-24ea-4b43-bd45-a64b38513b06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:59:30.176838 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-701933b0-24ea-4b43-bd45-a64b38513b06 None None] MFA Rules not processed for user `7634e72625254c12bc7427200135c55e`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:59:30.178012 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-701933b0-24ea-4b43-bd45-a64b38513b06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:59:30.239718 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1389/2762] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:59:30 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 19:59:30.244012 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23bd856-b826-4734-a03b-b8709bd8c684 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:59:30.304016 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23bd856-b826-4734-a03b-b8709bd8c684 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:59:30.308726 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64274f9-fc00-4a10-8c3c-d790f79e6174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:59:30.309378 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64274f9-fc00-4a10-8c3c-d790f79e6174 admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:59:30.309665 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64274f9-fc00-4a10-8c3c-d790f79e6174 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:59:30.309943 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64274f9-fc00-4a10-8c3c-d790f79e6174 admin admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:59:30.310632 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64274f9-fc00-4a10-8c3c-d790f79e6174 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:59:30.312550 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64274f9-fc00-4a10-8c3c-d790f79e6174 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:59:30.390150 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:59:30.397623 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1374/2763] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 19:59:30 2022] POST /identity/v3/projects => generated 373 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:59:30.401923 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701933b0-24ea-4b43-bd45-a64b38513b06 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:59:30.413035 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701933b0-24ea-4b43-bd45-a64b38513b06 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:59:30.460612 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:59:30.469402 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7257010-11e2-436d-9d09-26f1266122c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:59:30.470074 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7257010-11e2-436d-9d09-26f1266122c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:59:30.470394 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7257010-11e2-436d-9d09-26f1266122c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:59:30.471346 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7257010-11e2-436d-9d09-26f1266122c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:59:30.472050 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7257010-11e2-436d-9d09-26f1266122c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:59:30.473979 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7257010-11e2-436d-9d09-26f1266122c5 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:59:30.603258 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1390/2764] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 19:59:30 2022] POST /identity/v3/users => generated 293 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 19:59:30.607783 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64274f9-fc00-4a10-8c3c-d790f79e6174 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:59:30.618249 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64274f9-fc00-4a10-8c3c-d790f79e6174 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:59:30.622737 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbb0d43-fd2b-4970-869e-c89c6dd71f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:59:30.623447 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb0d43-fd2b-4970-869e-c89c6dd71f19 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:59:30.623731 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb0d43-fd2b-4970-869e-c89c6dd71f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:59:30.624051 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb0d43-fd2b-4970-869e-c89c6dd71f19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:59:30.624782 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbb0d43-fd2b-4970-869e-c89c6dd71f19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:59:30.626386 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbb0d43-fd2b-4970-869e-c89c6dd71f19 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:59:30.636676 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1375/2765] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:59:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:59:30.640544 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7257010-11e2-436d-9d09-26f1266122c5 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:59:30.651450 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7257010-11e2-436d-9d09-26f1266122c5 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:59:30.656564 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b26868-0da3-411b-bc71-f32906a1fb2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:59:30.657442 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b26868-0da3-411b-bc71-f32906a1fb2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:59:30.657732 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b26868-0da3-411b-bc71-f32906a1fb2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:59:30.658014 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b26868-0da3-411b-bc71-f32906a1fb2a admin admin] PATH_INFO: `/v3/projects/5f7d1b53075343f4a904c645b9a15642/users/0d6b102a2b224baca54eb05d93f363e3/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:59:30.673021 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b26868-0da3-411b-bc71-f32906a1fb2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f7d1b53075343f4a904c645b9a15642, user_id=0d6b102a2b224baca54eb05d93f363e3, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:59:30.675739 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b26868-0da3-411b-bc71-f32906a1fb2a admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:59:30.739393 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1391/2766] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:59:30 2022] PUT /identity/v3/projects/5f7d1b53075343f4a904c645b9a15642/users/0d6b102a2b224baca54eb05d93f363e3/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:59:30.743298 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbb0d43-fd2b-4970-869e-c89c6dd71f19 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:59:30.753485 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbb0d43-fd2b-4970-869e-c89c6dd71f19 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:59:30.806296 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c33c175-3dbe-445a-b6fa-b01e45614c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:59:30.806973 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c33c175-3dbe-445a-b6fa-b01e45614c33 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:59:30.807253 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c33c175-3dbe-445a-b6fa-b01e45614c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:59:30.807526 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c33c175-3dbe-445a-b6fa-b01e45614c33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:59:30.808235 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c33c175-3dbe-445a-b6fa-b01e45614c33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:59:30.809803 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c33c175-3dbe-445a-b6fa-b01e45614c33 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:59:30.819699 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1376/2767] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 19:59:30 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 19:59:30.823715 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b26868-0da3-411b-bc71-f32906a1fb2a admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:59:30.834561 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b26868-0da3-411b-bc71-f32906a1fb2a admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:59:30.839181 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d4136d-6700-42dc-931f-2db0e9c461ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:59:30.840061 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d4136d-6700-42dc-931f-2db0e9c461ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:59:30.840350 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d4136d-6700-42dc-931f-2db0e9c461ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:59:30.840626 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d4136d-6700-42dc-931f-2db0e9c461ac admin admin] PATH_INFO: `/v3/projects/5f7d1b53075343f4a904c645b9a15642/users/0d6b102a2b224baca54eb05d93f363e3/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:59:30.842589 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d4136d-6700-42dc-931f-2db0e9c461ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f7d1b53075343f4a904c645b9a15642, user_id=0d6b102a2b224baca54eb05d93f363e3, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:59:30.845234 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d4136d-6700-42dc-931f-2db0e9c461ac admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:59:30.855726 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1392/2768] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 19:59:30 2022] PUT /identity/v3/projects/5f7d1b53075343f4a904c645b9a15642/users/0d6b102a2b224baca54eb05d93f363e3/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 19:59:30.865693 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ce1cfa-fc1b-4823-91a1-508660c76531 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:59:30.865991 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ce1cfa-fc1b-4823-91a1-508660c76531 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:59:30.866278 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ce1cfa-fc1b-4823-91a1-508660c76531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:59:30.892240 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-93ce1cfa-fc1b-4823-91a1-508660c76531 None None] MFA Rules not processed for user `0d6b102a2b224baca54eb05d93f363e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 19:59:30.893987 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-93ce1cfa-fc1b-4823-91a1-508660c76531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:59:30.959071 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1377/2769] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 19:59:30 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 19:59:30.969913 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d4136d-6700-42dc-931f-2db0e9c461ac admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:59:30.971659 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-03d4136d-6700-42dc-931f-2db0e9c461ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:59:31.029086 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d4136d-6700-42dc-931f-2db0e9c461ac admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:59:31.032785 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749aa9dc-0686-411c-85e8-836097e66a0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:59:31.033429 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749aa9dc-0686-411c-85e8-836097e66a0a service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:59:31.033712 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749aa9dc-0686-411c-85e8-836097e66a0a service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:59:31.033988 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749aa9dc-0686-411c-85e8-836097e66a0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:59:31.042974 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749aa9dc-0686-411c-85e8-836097e66a0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:59:31.044403 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749aa9dc-0686-411c-85e8-836097e66a0a service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:59:31.053703 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1393/2770] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:59:30 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:59:31.249958 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ce1cfa-fc1b-4823-91a1-508660c76531 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:59:31.260128 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ce1cfa-fc1b-4823-91a1-508660c76531 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:59:31.263802 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58bd706-717b-4709-8b08-d3b780aee3df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:59:31.264546 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58bd706-717b-4709-8b08-d3b780aee3df service cinder] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:59:31.264834 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58bd706-717b-4709-8b08-d3b780aee3df service cinder] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:59:31.265115 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58bd706-717b-4709-8b08-d3b780aee3df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:59:31.321180 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58bd706-717b-4709-8b08-d3b780aee3df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:59:31.322638 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58bd706-717b-4709-8b08-d3b780aee3df service cinder] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:59:31.331448 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1378/2771] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:59:31 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:59:33.008338 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749aa9dc-0686-411c-85e8-836097e66a0a service cinder] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:59:33.010238 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-749aa9dc-0686-411c-85e8-836097e66a0a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:59:33.070647 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749aa9dc-0686-411c-85e8-836097e66a0a service cinder] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:59:33.074419 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dca7ad-8279-428f-8bd5-655ec7bd89d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:59:33.075116 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dca7ad-8279-428f-8bd5-655ec7bd89d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:59:33.075406 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dca7ad-8279-428f-8bd5-655ec7bd89d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:59:33.075689 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dca7ad-8279-428f-8bd5-655ec7bd89d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:59:33.077510 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-96dca7ad-8279-428f-8bd5-655ec7bd89d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:59:33.137899 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:59:33.152247 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dca7ad-8279-428f-8bd5-655ec7bd89d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:59:33.153697 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dca7ad-8279-428f-8bd5-655ec7bd89d2 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:59:33.165156 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1394/2772] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:59:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:59:41.327413 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=97 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:59:43.349106 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=5 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 19:59:52.520026 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58bd706-717b-4709-8b08-d3b780aee3df service cinder] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:59:52.521587 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-f58bd706-717b-4709-8b08-d3b780aee3df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:59:52.533862 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58bd706-717b-4709-8b08-d3b780aee3df service cinder] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:59:52.537433 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a7a615-f5bc-4ddb-8988-240336ec4337 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:59:52.538083 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a7a615-f5bc-4ddb-8988-240336ec4337 service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:59:52.538374 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a7a615-f5bc-4ddb-8988-240336ec4337 service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:59:52.538653 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a7a615-f5bc-4ddb-8988-240336ec4337 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:59:52.540485 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-c8a7a615-f5bc-4ddb-8988-240336ec4337 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:59:52.550279 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a7a615-f5bc-4ddb-8988-240336ec4337 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:59:52.551858 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a7a615-f5bc-4ddb-8988-240336ec4337 service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:59:52.560103 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1379/2773] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:59:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 19:59:52.645801 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dca7ad-8279-428f-8bd5-655ec7bd89d2 service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 19:59:52.654510 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dca7ad-8279-428f-8bd5-655ec7bd89d2 service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 19:59:52.658734 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391dc09e-63bb-4f7b-b09d-3eaa2ce41c11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 19:59:52.659398 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391dc09e-63bb-4f7b-b09d-3eaa2ce41c11 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 19:59:52.659681 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391dc09e-63bb-4f7b-b09d-3eaa2ce41c11 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 19:59:52.659958 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391dc09e-63bb-4f7b-b09d-3eaa2ce41c11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 19:59:52.661726 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-391dc09e-63bb-4f7b-b09d-3eaa2ce41c11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 19:59:52.675345 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391dc09e-63bb-4f7b-b09d-3eaa2ce41c11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 19:59:52.676842 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391dc09e-63bb-4f7b-b09d-3eaa2ce41c11 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 19:59:52.687542 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1395/2774] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 19:59:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 20:00:02.557087 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=6 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 20:00:02.682926 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=6 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 20:00:15.581380 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a7a615-f5bc-4ddb-8988-240336ec4337 service neutron] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:15.582924 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-c8a7a615-f5bc-4ddb-8988-240336ec4337 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:00:15.650471 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a7a615-f5bc-4ddb-8988-240336ec4337 service neutron] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:15.655407 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ac78ca-095c-4da1-8013-10e2e579ac99 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73ae837e99764bebbe7a6e06ac9049e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:15.656049 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ac78ca-095c-4da1-8013-10e2e579ac99 service placement] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:15.656861 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ac78ca-095c-4da1-8013-10e2e579ac99 service placement] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:15.657136 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ac78ca-095c-4da1-8013-10e2e579ac99 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:15.658908 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-24ac78ca-095c-4da1-8013-10e2e579ac99 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:00:15.669912 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ac78ca-095c-4da1-8013-10e2e579ac99 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:15.671517 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ac78ca-095c-4da1-8013-10e2e579ac99 service placement] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:15.681759 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1380/2775] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 20:00:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 20:00:18.091070 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391dc09e-63bb-4f7b-b09d-3eaa2ce41c11 service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:18.157339 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391dc09e-63bb-4f7b-b09d-3eaa2ce41c11 service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:18.162610 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2bda68-3a0b-4b23-bfb4-d5731ee18410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:18.163527 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2bda68-3a0b-4b23-bfb4-d5731ee18410 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:18.163816 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2bda68-3a0b-4b23-bfb4-d5731ee18410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:18.164091 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2bda68-3a0b-4b23-bfb4-d5731ee18410 admin admin] PATH_INFO: `/v3/users/7634e72625254c12bc7427200135c55e` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:18.165282 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2bda68-3a0b-4b23-bfb4-d5731ee18410 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7634e72625254c12bc7427200135c55e)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:18.167708 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2bda68-3a0b-4b23-bfb4-d5731ee18410 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:18.347261 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-cc2bda68-3a0b-4b23-bfb4-d5731ee18410 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7634e72625254c12bc7427200135c55e'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:18.354441 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-cc2bda68-3a0b-4b23-bfb4-d5731ee18410 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7634e72625254c12bc7427200135c55e'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:18.402172 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-cc2bda68-3a0b-4b23-bfb4-d5731ee18410 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7634e72625254c12bc7427200135c55e'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:18.402886 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-cc2bda68-3a0b-4b23-bfb4-d5731ee18410 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7634e72625254c12bc7427200135c55e'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:18.403509 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-cc2bda68-3a0b-4b23-bfb4-d5731ee18410 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7634e72625254c12bc7427200135c55e'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:18.421399 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1396/2776] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 20:00:18 2022] DELETE /identity/v3/users/7634e72625254c12bc7427200135c55e => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 20:00:18.426481 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3f66b3-3f5e-4a76-b3ef-f6b90eaf7910 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:18.426974 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3f66b3-3f5e-4a76-b3ef-f6b90eaf7910 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:18.427071 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3f66b3-3f5e-4a76-b3ef-f6b90eaf7910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:18.444321 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-2d3f66b3-3f5e-4a76-b3ef-f6b90eaf7910 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 20:00:18.445518 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-2d3f66b3-3f5e-4a76-b3ef-f6b90eaf7910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:00:18.502216 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1381/2777] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 20:00:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 20:00:18.513376 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2bda68-3a0b-4b23-bfb4-d5731ee18410 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:18.514876 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-cc2bda68-3a0b-4b23-bfb4-d5731ee18410 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:00:18.571435 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2bda68-3a0b-4b23-bfb4-d5731ee18410 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:18.575072 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde90737-a389-4fd2-86db-3fc59b5a09ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:18.575718 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde90737-a389-4fd2-86db-3fc59b5a09ce service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:18.576006 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde90737-a389-4fd2-86db-3fc59b5a09ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:18.576285 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde90737-a389-4fd2-86db-3fc59b5a09ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:18.585310 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde90737-a389-4fd2-86db-3fc59b5a09ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:18.586761 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde90737-a389-4fd2-86db-3fc59b5a09ce service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:18.596726 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1397/2778] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 20:00:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 20:00:18.976051 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3f66b3-3f5e-4a76-b3ef-f6b90eaf7910 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:18.977491 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-2d3f66b3-3f5e-4a76-b3ef-f6b90eaf7910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:00:18.992498 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3f66b3-3f5e-4a76-b3ef-f6b90eaf7910 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:18.997228 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dabbd84-bd5e-4386-8007-ad57226be248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:18.998984 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dabbd84-bd5e-4386-8007-ad57226be248 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:18.999298 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dabbd84-bd5e-4386-8007-ad57226be248 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:18.999542 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dabbd84-bd5e-4386-8007-ad57226be248 admin admin] PATH_INFO: `/v3/projects/abbbb1aa51c246eca467a32562ffd7ce` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:19.000740 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dabbd84-bd5e-4386-8007-ad57226be248 admin admin] RBAC: Authorizing `identity:delete_project(project_id=abbbb1aa51c246eca467a32562ffd7ce)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:19.002651 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dabbd84-bd5e-4386-8007-ad57226be248 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:19.104999 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-4dabbd84-bd5e-4386-8007-ad57226be248 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abbbb1aa51c246eca467a32562ffd7ce', 'request_id': 'req-4dabbd84-bd5e-4386-8007-ad57226be248'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:19.166699 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-4dabbd84-bd5e-4386-8007-ad57226be248 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abbbb1aa51c246eca467a32562ffd7ce', 'request_id': 'req-4dabbd84-bd5e-4386-8007-ad57226be248'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:19.173183 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-4dabbd84-bd5e-4386-8007-ad57226be248 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abbbb1aa51c246eca467a32562ffd7ce', 'request_id': 'req-4dabbd84-bd5e-4386-8007-ad57226be248'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:19.184752 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-4dabbd84-bd5e-4386-8007-ad57226be248 admin admin] The token cache is being invalidate because project abbbb1aa51c246eca467a32562ffd7ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 20:00:19.185044 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-4dabbd84-bd5e-4386-8007-ad57226be248 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:19.185685 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-4dabbd84-bd5e-4386-8007-ad57226be248 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:19.187257 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1382/2779] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 20:00:18 2022] DELETE /identity/v3/projects/abbbb1aa51c246eca467a32562ffd7ce => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 20:00:19.191490 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde90737-a389-4fd2-86db-3fc59b5a09ce service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:19.201177 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde90737-a389-4fd2-86db-3fc59b5a09ce service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:19.219385 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 20:00:19.257010 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89412f18-4043-4c36-8a4c-908266ca5098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:19.257880 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89412f18-4043-4c36-8a4c-908266ca5098 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:19.258165 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89412f18-4043-4c36-8a4c-908266ca5098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:19.258440 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89412f18-4043-4c36-8a4c-908266ca5098 admin admin] PATH_INFO: `/v3/users/0d6b102a2b224baca54eb05d93f363e3` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:19.259676 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89412f18-4043-4c36-8a4c-908266ca5098 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d6b102a2b224baca54eb05d93f363e3)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:19.262325 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89412f18-4043-4c36-8a4c-908266ca5098 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:19.461246 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-89412f18-4043-4c36-8a4c-908266ca5098 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d6b102a2b224baca54eb05d93f363e3'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:19.467988 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-89412f18-4043-4c36-8a4c-908266ca5098 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d6b102a2b224baca54eb05d93f363e3'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:19.544840 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-89412f18-4043-4c36-8a4c-908266ca5098 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d6b102a2b224baca54eb05d93f363e3'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:19.545575 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-89412f18-4043-4c36-8a4c-908266ca5098 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d6b102a2b224baca54eb05d93f363e3'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:19.546277 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-89412f18-4043-4c36-8a4c-908266ca5098 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d6b102a2b224baca54eb05d93f363e3'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:19.564042 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1398/2780] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 20:00:19 2022] DELETE /identity/v3/users/0d6b102a2b224baca54eb05d93f363e3 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 20:00:20.003065 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dabbd84-bd5e-4386-8007-ad57226be248 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:20.004504 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-4dabbd84-bd5e-4386-8007-ad57226be248 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:00:20.017175 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dabbd84-bd5e-4386-8007-ad57226be248 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:20.026660 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 20:00:20.075079 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07218d0-f3bf-4565-93c5-8f8822098187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:20.075949 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07218d0-f3bf-4565-93c5-8f8822098187 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:20.076239 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07218d0-f3bf-4565-93c5-8f8822098187 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:20.076522 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07218d0-f3bf-4565-93c5-8f8822098187 admin admin] PATH_INFO: `/v3/projects/5f7d1b53075343f4a904c645b9a15642` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:20.077706 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07218d0-f3bf-4565-93c5-8f8822098187 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f7d1b53075343f4a904c645b9a15642)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:20.079729 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07218d0-f3bf-4565-93c5-8f8822098187 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:20.196571 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-b07218d0-f3bf-4565-93c5-8f8822098187 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f7d1b53075343f4a904c645b9a15642', 'request_id': 'req-b07218d0-f3bf-4565-93c5-8f8822098187'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:20.241833 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-b07218d0-f3bf-4565-93c5-8f8822098187 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f7d1b53075343f4a904c645b9a15642', 'request_id': 'req-b07218d0-f3bf-4565-93c5-8f8822098187'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:20.247914 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-b07218d0-f3bf-4565-93c5-8f8822098187 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f7d1b53075343f4a904c645b9a15642', 'request_id': 'req-b07218d0-f3bf-4565-93c5-8f8822098187'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:20.258558 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-b07218d0-f3bf-4565-93c5-8f8822098187 admin admin] The token cache is being invalidate because project 5f7d1b53075343f4a904c645b9a15642 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 20:00:20.258877 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-b07218d0-f3bf-4565-93c5-8f8822098187 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:20.259510 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-b07218d0-f3bf-4565-93c5-8f8822098187 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:20.261059 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1383/2781] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 20:00:20 2022] DELETE /identity/v3/projects/5f7d1b53075343f4a904c645b9a15642 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 20:00:20.265919 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99bcb4c-1263-48f6-babc-d8036a5c3b27 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:20.266385 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99bcb4c-1263-48f6-babc-d8036a5c3b27 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:20.266488 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99bcb4c-1263-48f6-babc-d8036a5c3b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:20.284191 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-f99bcb4c-1263-48f6-babc-d8036a5c3b27 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 20:00:20.285385 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-f99bcb4c-1263-48f6-babc-d8036a5c3b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:00:20.344329 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1399/2782] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 20:00:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 20:00:20.352812 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d8c1d5-5a76-4aa4-9190-e2b069237f04 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:20.353099 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d8c1d5-5a76-4aa4-9190-e2b069237f04 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:20.353374 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d8c1d5-5a76-4aa4-9190-e2b069237f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:20.371483 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-41d8c1d5-5a76-4aa4-9190-e2b069237f04 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 20:00:20.372656 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-41d8c1d5-5a76-4aa4-9190-e2b069237f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:00:20.380286 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1384/2783] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 20:00:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 20:00:20.384143 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99bcb4c-1263-48f6-babc-d8036a5c3b27 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:20.393448 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99bcb4c-1263-48f6-babc-d8036a5c3b27 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:20.397876 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44138425-a397-4291-a507-45e558b262ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:20.398634 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44138425-a397-4291-a507-45e558b262ed admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:20.398933 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44138425-a397-4291-a507-45e558b262ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:20.399212 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44138425-a397-4291-a507-45e558b262ed admin admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:20.399937 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44138425-a397-4291-a507-45e558b262ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 20:00:20.400245 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44138425-a397-4291-a507-45e558b262ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:20.401842 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44138425-a397-4291-a507-45e558b262ed admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:20.414154 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1400/2784] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 2 20:00:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 20:00:20.418291 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d8c1d5-5a76-4aa4-9190-e2b069237f04 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:20.428038 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d8c1d5-5a76-4aa4-9190-e2b069237f04 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:20.432689 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f17db4-bcfe-4272-9fbf-81ad06537a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:20.433335 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f17db4-bcfe-4272-9fbf-81ad06537a76 admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:20.433622 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f17db4-bcfe-4272-9fbf-81ad06537a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:20.433899 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f17db4-bcfe-4272-9fbf-81ad06537a76 admin admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:20.434623 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f17db4-bcfe-4272-9fbf-81ad06537a76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:20.436559 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f17db4-bcfe-4272-9fbf-81ad06537a76 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:20.517462 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1385/2785] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 20:00:20 2022] POST /identity/v3/projects => generated 355 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 20:00:20.522094 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44138425-a397-4291-a507-45e558b262ed admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:20.532059 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44138425-a397-4291-a507-45e558b262ed admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:20.558045 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 20:00:20.589039 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ba7005-60ae-4557-a951-a26ecbaeef7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:20.589707 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ba7005-60ae-4557-a951-a26ecbaeef7b admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:20.590033 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ba7005-60ae-4557-a951-a26ecbaeef7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:20.590338 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ba7005-60ae-4557-a951-a26ecbaeef7b admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:20.591075 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ba7005-60ae-4557-a951-a26ecbaeef7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:20.593037 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ba7005-60ae-4557-a951-a26ecbaeef7b admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:20.670200 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1401/2786] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 20:00:20 2022] POST /identity/v3/users => generated 278 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 20:00:20.674309 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f17db4-bcfe-4272-9fbf-81ad06537a76 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:20.684584 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f17db4-bcfe-4272-9fbf-81ad06537a76 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:20.689185 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf686a2d-a42a-4c9c-adb0-06f388fff810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:20.689890 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf686a2d-a42a-4c9c-adb0-06f388fff810 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:20.690174 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf686a2d-a42a-4c9c-adb0-06f388fff810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:20.690452 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf686a2d-a42a-4c9c-adb0-06f388fff810 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:20.691214 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf686a2d-a42a-4c9c-adb0-06f388fff810 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:20.692796 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf686a2d-a42a-4c9c-adb0-06f388fff810 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:20.702879 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1386/2787] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 20:00:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 20:00:20.707336 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ba7005-60ae-4557-a951-a26ecbaeef7b admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:20.717553 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ba7005-60ae-4557-a951-a26ecbaeef7b admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:20.722866 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01062d6c-68ff-41eb-a9c1-20a9b5c2c40e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:20.723520 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01062d6c-68ff-41eb-a9c1-20a9b5c2c40e admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:20.723804 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01062d6c-68ff-41eb-a9c1-20a9b5c2c40e admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:20.724081 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01062d6c-68ff-41eb-a9c1-20a9b5c2c40e admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:20.724806 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01062d6c-68ff-41eb-a9c1-20a9b5c2c40e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:20.726657 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01062d6c-68ff-41eb-a9c1-20a9b5c2c40e admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:20.737037 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1402/2788] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 20:00:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 20:00:20.740984 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf686a2d-a42a-4c9c-adb0-06f388fff810 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:20.750550 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf686a2d-a42a-4c9c-adb0-06f388fff810 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:20.755685 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d6ec1d-0330-4e69-81cd-0095a5de434f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:20.756545 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6ec1d-0330-4e69-81cd-0095a5de434f admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:20.756825 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6ec1d-0330-4e69-81cd-0095a5de434f admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:20.757098 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6ec1d-0330-4e69-81cd-0095a5de434f admin admin] PATH_INFO: `/v3/projects/900189aa8bf443799c4aaeeb8289f182/users/1a1bb84b1209445abd15691c2925445f/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:20.771582 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d6ec1d-0330-4e69-81cd-0095a5de434f admin admin] RBAC: Authorizing `identity:create_grant(project_id=900189aa8bf443799c4aaeeb8289f182, user_id=1a1bb84b1209445abd15691c2925445f, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:20.774150 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d6ec1d-0330-4e69-81cd-0095a5de434f admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:20.841585 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1387/2789] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 20:00:20 2022] PUT /identity/v3/projects/900189aa8bf443799c4aaeeb8289f182/users/1a1bb84b1209445abd15691c2925445f/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 20:00:20.853304 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132dea6f-c64b-44f6-8b68-ef9eb7ed6af6 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:20.853586 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132dea6f-c64b-44f6-8b68-ef9eb7ed6af6 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:20.853869 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132dea6f-c64b-44f6-8b68-ef9eb7ed6af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:20.880507 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-132dea6f-c64b-44f6-8b68-ef9eb7ed6af6 None None] MFA Rules not processed for user `1a1bb84b1209445abd15691c2925445f`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 20:00:20.881733 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-132dea6f-c64b-44f6-8b68-ef9eb7ed6af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:00:20.945243 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1403/2790] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 20:00:20 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 20:00:20.949551 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d6ec1d-0330-4e69-81cd-0095a5de434f admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:20.990587 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 20:00:21.009757 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d6ec1d-0330-4e69-81cd-0095a5de434f admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:21.014518 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c37c643-9a3f-48d5-a4b6-35857e16ea15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:21.015208 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c37c643-9a3f-48d5-a4b6-35857e16ea15 admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:21.015499 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c37c643-9a3f-48d5-a4b6-35857e16ea15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:21.015780 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c37c643-9a3f-48d5-a4b6-35857e16ea15 admin admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:21.016478 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c37c643-9a3f-48d5-a4b6-35857e16ea15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:21.018981 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c37c643-9a3f-48d5-a4b6-35857e16ea15 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:21.110284 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1388/2791] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 20:00:20 2022] POST /identity/v3/projects => generated 355 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 20:00:21.114577 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132dea6f-c64b-44f6-8b68-ef9eb7ed6af6 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:21.125599 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132dea6f-c64b-44f6-8b68-ef9eb7ed6af6 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:21.182539 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac190c5-77d2-4638-8326-8dfa2b01b61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:21.183242 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac190c5-77d2-4638-8326-8dfa2b01b61c admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:21.183532 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac190c5-77d2-4638-8326-8dfa2b01b61c admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:21.183814 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac190c5-77d2-4638-8326-8dfa2b01b61c admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:21.184542 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac190c5-77d2-4638-8326-8dfa2b01b61c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:21.186474 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac190c5-77d2-4638-8326-8dfa2b01b61c admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:21.259285 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1404/2792] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 20:00:21 2022] POST /identity/v3/users => generated 284 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 20:00:21.263199 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c37c643-9a3f-48d5-a4b6-35857e16ea15 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:21.273403 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c37c643-9a3f-48d5-a4b6-35857e16ea15 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:21.277909 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867f4eba-0139-4d55-b6ba-294742a99a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:21.278557 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867f4eba-0139-4d55-b6ba-294742a99a30 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:21.278865 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867f4eba-0139-4d55-b6ba-294742a99a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:21.279144 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867f4eba-0139-4d55-b6ba-294742a99a30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:21.279856 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867f4eba-0139-4d55-b6ba-294742a99a30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:21.281438 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867f4eba-0139-4d55-b6ba-294742a99a30 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:21.291647 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1389/2793] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 20:00:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 20:00:21.295600 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac190c5-77d2-4638-8326-8dfa2b01b61c admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:21.306143 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac190c5-77d2-4638-8326-8dfa2b01b61c admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:21.310956 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6270be6-6640-4ae7-bae5-6fec2fda6eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:21.311843 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6270be6-6640-4ae7-bae5-6fec2fda6eaf admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:21.312135 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6270be6-6640-4ae7-bae5-6fec2fda6eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:21.312417 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6270be6-6640-4ae7-bae5-6fec2fda6eaf admin admin] PATH_INFO: `/v3/projects/e40b5d15d1784b46b79f378d56727ff6/users/6698b284b2ca4cebb677fadd7844caa7/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:21.328176 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6270be6-6640-4ae7-bae5-6fec2fda6eaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=e40b5d15d1784b46b79f378d56727ff6, user_id=6698b284b2ca4cebb677fadd7844caa7, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:21.330922 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6270be6-6640-4ae7-bae5-6fec2fda6eaf admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:21.407152 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1405/2794] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 20:00:21 2022] PUT /identity/v3/projects/e40b5d15d1784b46b79f378d56727ff6/users/6698b284b2ca4cebb677fadd7844caa7/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 20:00:21.411019 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867f4eba-0139-4d55-b6ba-294742a99a30 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:21.420920 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867f4eba-0139-4d55-b6ba-294742a99a30 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:21.476174 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae17628-8701-49bb-9feb-d112806124a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:21.476836 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae17628-8701-49bb-9feb-d112806124a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:21.477125 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae17628-8701-49bb-9feb-d112806124a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:21.477412 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae17628-8701-49bb-9feb-d112806124a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:21.478153 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae17628-8701-49bb-9feb-d112806124a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:21.479743 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae17628-8701-49bb-9feb-d112806124a5 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:21.489827 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1390/2795] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 20:00:21 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 20:00:21.494197 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6270be6-6640-4ae7-bae5-6fec2fda6eaf admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:21.499729 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 20:00:21.505625 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6270be6-6640-4ae7-bae5-6fec2fda6eaf admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:21.510261 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ccb6b6-6b17-4db8-8bd9-8ade3ff0ee45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:21.511221 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ccb6b6-6b17-4db8-8bd9-8ade3ff0ee45 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:21.511512 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ccb6b6-6b17-4db8-8bd9-8ade3ff0ee45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:21.511793 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ccb6b6-6b17-4db8-8bd9-8ade3ff0ee45 admin admin] PATH_INFO: `/v3/projects/e40b5d15d1784b46b79f378d56727ff6/users/6698b284b2ca4cebb677fadd7844caa7/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:21.513758 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ccb6b6-6b17-4db8-8bd9-8ade3ff0ee45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e40b5d15d1784b46b79f378d56727ff6, user_id=6698b284b2ca4cebb677fadd7844caa7, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:21.516475 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ccb6b6-6b17-4db8-8bd9-8ade3ff0ee45 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:21.526816 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1406/2796] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 20:00:21 2022] PUT /identity/v3/projects/e40b5d15d1784b46b79f378d56727ff6/users/6698b284b2ca4cebb677fadd7844caa7/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 20:00:21.536724 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63656ce-043e-4dc1-833d-5386b9e2c3ac None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:21.537013 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63656ce-043e-4dc1-833d-5386b9e2c3ac None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:21.537290 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63656ce-043e-4dc1-833d-5386b9e2c3ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:21.563522 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-f63656ce-043e-4dc1-833d-5386b9e2c3ac None None] MFA Rules not processed for user `6698b284b2ca4cebb677fadd7844caa7`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 20:00:21.565333 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-f63656ce-043e-4dc1-833d-5386b9e2c3ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:00:21.632213 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1391/2797] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 20:00:21 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 20:00:21.643689 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ccb6b6-6b17-4db8-8bd9-8ade3ff0ee45 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:21.645422 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-05ccb6b6-6b17-4db8-8bd9-8ade3ff0ee45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:00:21.702435 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ccb6b6-6b17-4db8-8bd9-8ade3ff0ee45 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:21.706110 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d4fbd5-f8fb-451c-b22c-acfc842b20ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:21.706763 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4fbd5-f8fb-451c-b22c-acfc842b20ed service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:21.707069 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4fbd5-f8fb-451c-b22c-acfc842b20ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:21.707350 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4fbd5-f8fb-451c-b22c-acfc842b20ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:21.716232 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d4fbd5-f8fb-451c-b22c-acfc842b20ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:21.717674 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d4fbd5-f8fb-451c-b22c-acfc842b20ed service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:21.727598 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1407/2798] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 20:00:21 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 20:00:21.962923 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63656ce-043e-4dc1-833d-5386b9e2c3ac None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:21.973271 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63656ce-043e-4dc1-833d-5386b9e2c3ac None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:21.977393 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3059207-cda1-48f7-b953-971780129542 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:21.978069 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3059207-cda1-48f7-b953-971780129542 service cinder] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:21.986418 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3059207-cda1-48f7-b953-971780129542 service cinder] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:21.986418 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3059207-cda1-48f7-b953-971780129542 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:22.010952 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 20:00:22.035560 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3059207-cda1-48f7-b953-971780129542 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:22.037000 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3059207-cda1-48f7-b953-971780129542 service cinder] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:22.045816 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1392/2799] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 20:00:21 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 20:00:31.723235 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 20:00:32.081739 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=12 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 20:00:39.360754 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d4fbd5-f8fb-451c-b22c-acfc842b20ed service cinder] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:39.422856 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d4fbd5-f8fb-451c-b22c-acfc842b20ed service cinder] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:39.428012 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4fb833-7f5e-402a-8e8e-95faaca5004f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:39.428884 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4fb833-7f5e-402a-8e8e-95faaca5004f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:39.429172 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4fb833-7f5e-402a-8e8e-95faaca5004f admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:39.429454 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4fb833-7f5e-402a-8e8e-95faaca5004f admin admin] PATH_INFO: `/v3/users/1a1bb84b1209445abd15691c2925445f` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:39.430634 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4fb833-7f5e-402a-8e8e-95faaca5004f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a1bb84b1209445abd15691c2925445f)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:39.432687 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4fb833-7f5e-402a-8e8e-95faaca5004f admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:39.612156 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-ce4fb833-7f5e-402a-8e8e-95faaca5004f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a1bb84b1209445abd15691c2925445f'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:39.619140 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-ce4fb833-7f5e-402a-8e8e-95faaca5004f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a1bb84b1209445abd15691c2925445f'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:39.685259 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-ce4fb833-7f5e-402a-8e8e-95faaca5004f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a1bb84b1209445abd15691c2925445f'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:39.685876 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-ce4fb833-7f5e-402a-8e8e-95faaca5004f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a1bb84b1209445abd15691c2925445f'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:39.686511 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-ce4fb833-7f5e-402a-8e8e-95faaca5004f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a1bb84b1209445abd15691c2925445f'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:39.703812 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1408/2800] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 20:00:39 2022] DELETE /identity/v3/users/1a1bb84b1209445abd15691c2925445f => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 20:00:39.708300 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d67ec1-b6df-46bc-8fa9-84c4753e0ef1 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:39.708779 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d67ec1-b6df-46bc-8fa9-84c4753e0ef1 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:39.708889 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d67ec1-b6df-46bc-8fa9-84c4753e0ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:39.726304 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-48d67ec1-b6df-46bc-8fa9-84c4753e0ef1 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 20:00:39.727533 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-48d67ec1-b6df-46bc-8fa9-84c4753e0ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:00:39.785010 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1393/2801] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 20:00:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 20:00:39.794435 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4fb833-7f5e-402a-8e8e-95faaca5004f admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:39.795867 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-ce4fb833-7f5e-402a-8e8e-95faaca5004f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:00:39.852260 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4fb833-7f5e-402a-8e8e-95faaca5004f admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:39.855902 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af66ae15-3a32-4bc7-8815-0f796a599f4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:39.856550 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af66ae15-3a32-4bc7-8815-0f796a599f4d service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:39.856842 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af66ae15-3a32-4bc7-8815-0f796a599f4d service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:39.857121 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af66ae15-3a32-4bc7-8815-0f796a599f4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:39.866674 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af66ae15-3a32-4bc7-8815-0f796a599f4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:39.868086 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af66ae15-3a32-4bc7-8815-0f796a599f4d service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:39.877466 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1409/2802] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 20:00:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 20:00:40.281067 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d67ec1-b6df-46bc-8fa9-84c4753e0ef1 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:40.282501 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-48d67ec1-b6df-46bc-8fa9-84c4753e0ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:00:40.295134 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d67ec1-b6df-46bc-8fa9-84c4753e0ef1 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:40.299775 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225b1b4e-238f-4547-a489-709b2a29bd6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:40.300640 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225b1b4e-238f-4547-a489-709b2a29bd6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:40.300931 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225b1b4e-238f-4547-a489-709b2a29bd6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:40.301213 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225b1b4e-238f-4547-a489-709b2a29bd6f admin admin] PATH_INFO: `/v3/projects/900189aa8bf443799c4aaeeb8289f182` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:40.302867 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225b1b4e-238f-4547-a489-709b2a29bd6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=900189aa8bf443799c4aaeeb8289f182)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:40.304761 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225b1b4e-238f-4547-a489-709b2a29bd6f admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:40.433434 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-225b1b4e-238f-4547-a489-709b2a29bd6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '900189aa8bf443799c4aaeeb8289f182', 'request_id': 'req-225b1b4e-238f-4547-a489-709b2a29bd6f'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:40.496185 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-225b1b4e-238f-4547-a489-709b2a29bd6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '900189aa8bf443799c4aaeeb8289f182', 'request_id': 'req-225b1b4e-238f-4547-a489-709b2a29bd6f'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:40.502418 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-225b1b4e-238f-4547-a489-709b2a29bd6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '900189aa8bf443799c4aaeeb8289f182', 'request_id': 'req-225b1b4e-238f-4547-a489-709b2a29bd6f'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:40.513445 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-225b1b4e-238f-4547-a489-709b2a29bd6f admin admin] The token cache is being invalidate because project 900189aa8bf443799c4aaeeb8289f182 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 20:00:40.513736 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-225b1b4e-238f-4547-a489-709b2a29bd6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:40.514366 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-225b1b4e-238f-4547-a489-709b2a29bd6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:40.515936 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1394/2803] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 20:00:40 2022] DELETE /identity/v3/projects/900189aa8bf443799c4aaeeb8289f182 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 20:00:40.520580 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af66ae15-3a32-4bc7-8815-0f796a599f4d service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:40.530103 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af66ae15-3a32-4bc7-8815-0f796a599f4d service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:40.553911 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 20:00:40.591851 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a487b6a7-8174-43ee-8bc7-9e877b0336bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:40.592730 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a487b6a7-8174-43ee-8bc7-9e877b0336bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:40.593008 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a487b6a7-8174-43ee-8bc7-9e877b0336bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:40.593283 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a487b6a7-8174-43ee-8bc7-9e877b0336bb admin admin] PATH_INFO: `/v3/users/6698b284b2ca4cebb677fadd7844caa7` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:40.594456 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a487b6a7-8174-43ee-8bc7-9e877b0336bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=6698b284b2ca4cebb677fadd7844caa7)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:40.596459 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a487b6a7-8174-43ee-8bc7-9e877b0336bb admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:40.744609 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-a487b6a7-8174-43ee-8bc7-9e877b0336bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6698b284b2ca4cebb677fadd7844caa7'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:40.751745 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-a487b6a7-8174-43ee-8bc7-9e877b0336bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6698b284b2ca4cebb677fadd7844caa7'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:40.810405 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-a487b6a7-8174-43ee-8bc7-9e877b0336bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6698b284b2ca4cebb677fadd7844caa7'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:40.811082 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-a487b6a7-8174-43ee-8bc7-9e877b0336bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6698b284b2ca4cebb677fadd7844caa7'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:40.811725 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-a487b6a7-8174-43ee-8bc7-9e877b0336bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6698b284b2ca4cebb677fadd7844caa7'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:40.829619 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1410/2804] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 20:00:40 2022] DELETE /identity/v3/users/6698b284b2ca4cebb677fadd7844caa7 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 20:00:41.258079 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225b1b4e-238f-4547-a489-709b2a29bd6f admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:41.259547 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-225b1b4e-238f-4547-a489-709b2a29bd6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:00:41.274586 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225b1b4e-238f-4547-a489-709b2a29bd6f admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:41.332197 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c0f787-79cc-45a0-aa6f-40bbc65a54f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:41.333053 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c0f787-79cc-45a0-aa6f-40bbc65a54f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:41.333338 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c0f787-79cc-45a0-aa6f-40bbc65a54f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:41.333617 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c0f787-79cc-45a0-aa6f-40bbc65a54f1 admin admin] PATH_INFO: `/v3/projects/e40b5d15d1784b46b79f378d56727ff6` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:41.334822 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c0f787-79cc-45a0-aa6f-40bbc65a54f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e40b5d15d1784b46b79f378d56727ff6)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:41.336702 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c0f787-79cc-45a0-aa6f-40bbc65a54f1 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:41.429180 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 20:00:41.445590 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-88c0f787-79cc-45a0-aa6f-40bbc65a54f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e40b5d15d1784b46b79f378d56727ff6', 'request_id': 'req-88c0f787-79cc-45a0-aa6f-40bbc65a54f1'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:41.507507 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-88c0f787-79cc-45a0-aa6f-40bbc65a54f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e40b5d15d1784b46b79f378d56727ff6', 'request_id': 'req-88c0f787-79cc-45a0-aa6f-40bbc65a54f1'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:41.513282 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-88c0f787-79cc-45a0-aa6f-40bbc65a54f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e40b5d15d1784b46b79f378d56727ff6', 'request_id': 'req-88c0f787-79cc-45a0-aa6f-40bbc65a54f1'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:41.524715 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-88c0f787-79cc-45a0-aa6f-40bbc65a54f1 admin admin] The token cache is being invalidate because project e40b5d15d1784b46b79f378d56727ff6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 20:00:41.525002 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-88c0f787-79cc-45a0-aa6f-40bbc65a54f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:41.525658 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-88c0f787-79cc-45a0-aa6f-40bbc65a54f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:41.527208 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1395/2805] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 20:00:41 2022] DELETE /identity/v3/projects/e40b5d15d1784b46b79f378d56727ff6 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 20:00:41.532220 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877f9a4c-33af-476f-abbc-70a6b3551a44 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:41.532691 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877f9a4c-33af-476f-abbc-70a6b3551a44 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:41.532803 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877f9a4c-33af-476f-abbc-70a6b3551a44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:41.550484 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-877f9a4c-33af-476f-abbc-70a6b3551a44 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 20:00:41.551718 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-877f9a4c-33af-476f-abbc-70a6b3551a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:00:41.609775 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1411/2806] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 20:00:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 20:00:41.618292 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b217141c-49ec-48bb-a2d1-2ba56674ece8 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:41.618589 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b217141c-49ec-48bb-a2d1-2ba56674ece8 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:41.618919 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b217141c-49ec-48bb-a2d1-2ba56674ece8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:41.637964 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-b217141c-49ec-48bb-a2d1-2ba56674ece8 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 20:00:41.639207 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-b217141c-49ec-48bb-a2d1-2ba56674ece8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:00:41.647256 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1396/2807] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 20:00:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 20:00:41.651356 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877f9a4c-33af-476f-abbc-70a6b3551a44 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:41.660837 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877f9a4c-33af-476f-abbc-70a6b3551a44 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:41.665389 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d795bd5-9d0d-427f-a8d2-f33a6e6dbf57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:41.666039 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d795bd5-9d0d-427f-a8d2-f33a6e6dbf57 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:41.666327 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d795bd5-9d0d-427f-a8d2-f33a6e6dbf57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:41.666610 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d795bd5-9d0d-427f-a8d2-f33a6e6dbf57 admin admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:41.667377 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d795bd5-9d0d-427f-a8d2-f33a6e6dbf57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 20:00:41.667687 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d795bd5-9d0d-427f-a8d2-f33a6e6dbf57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:41.669283 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d795bd5-9d0d-427f-a8d2-f33a6e6dbf57 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:41.681112 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1412/2808] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 2 20:00:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 20:00:41.685080 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b217141c-49ec-48bb-a2d1-2ba56674ece8 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:41.694637 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b217141c-49ec-48bb-a2d1-2ba56674ece8 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:41.699305 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33b04e4-c23d-4f94-a640-5bf519e1b8c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:41.699948 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33b04e4-c23d-4f94-a640-5bf519e1b8c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:41.700236 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33b04e4-c23d-4f94-a640-5bf519e1b8c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:41.700515 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33b04e4-c23d-4f94-a640-5bf519e1b8c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:41.701211 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33b04e4-c23d-4f94-a640-5bf519e1b8c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:41.703143 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33b04e4-c23d-4f94-a640-5bf519e1b8c5 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:41.776729 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1397/2809] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 20:00:41 2022] POST /identity/v3/projects => generated 399 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 02 20:00:41.780695 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d795bd5-9d0d-427f-a8d2-f33a6e6dbf57 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:41.791116 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d795bd5-9d0d-427f-a8d2-f33a6e6dbf57 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:41.818463 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 20:00:41.848946 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf3f22b-4744-4447-b855-c69b977e580e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:41.849682 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf3f22b-4744-4447-b855-c69b977e580e admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:41.849977 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf3f22b-4744-4447-b855-c69b977e580e admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:41.850260 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf3f22b-4744-4447-b855-c69b977e580e admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:41.850985 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf3f22b-4744-4447-b855-c69b977e580e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:41.852922 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf3f22b-4744-4447-b855-c69b977e580e admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:41.925650 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1413/2810] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 20:00:41 2022] POST /identity/v3/users => generated 300 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 20:00:41.929774 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33b04e4-c23d-4f94-a640-5bf519e1b8c5 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:41.940423 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33b04e4-c23d-4f94-a640-5bf519e1b8c5 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:41.945083 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19535b3-1a98-4255-987d-bd1f499f13e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:41.945858 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19535b3-1a98-4255-987d-bd1f499f13e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:41.946152 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19535b3-1a98-4255-987d-bd1f499f13e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:41.946436 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19535b3-1a98-4255-987d-bd1f499f13e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:41.947830 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19535b3-1a98-4255-987d-bd1f499f13e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:41.948914 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19535b3-1a98-4255-987d-bd1f499f13e0 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:41.959638 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1398/2811] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 20:00:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 20:00:41.963642 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf3f22b-4744-4447-b855-c69b977e580e admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:41.974411 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf3f22b-4744-4447-b855-c69b977e580e admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:41.979073 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1229b6-a417-40d5-90b9-8c0c7255d7f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:41.979734 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1229b6-a417-40d5-90b9-8c0c7255d7f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:41.980023 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1229b6-a417-40d5-90b9-8c0c7255d7f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:41.980308 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1229b6-a417-40d5-90b9-8c0c7255d7f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:41.981066 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1229b6-a417-40d5-90b9-8c0c7255d7f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:41.982652 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1229b6-a417-40d5-90b9-8c0c7255d7f3 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:41.995289 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1414/2812] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 20:00:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 20:00:41.999092 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19535b3-1a98-4255-987d-bd1f499f13e0 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:42.009103 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19535b3-1a98-4255-987d-bd1f499f13e0 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:42.013669 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ca3a21-27ae-442b-afd6-3b4ee14bedeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:42.014546 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca3a21-27ae-442b-afd6-3b4ee14bedeb admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:42.014852 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca3a21-27ae-442b-afd6-3b4ee14bedeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:42.015133 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca3a21-27ae-442b-afd6-3b4ee14bedeb admin admin] PATH_INFO: `/v3/projects/124df214b4854640b710007503ba2a33/users/9c38ae3bc3384cdba9654d5ebb362a79/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:42.030238 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ca3a21-27ae-442b-afd6-3b4ee14bedeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=124df214b4854640b710007503ba2a33, user_id=9c38ae3bc3384cdba9654d5ebb362a79, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:42.031850 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ca3a21-27ae-442b-afd6-3b4ee14bedeb admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:42.114036 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1399/2813] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 20:00:41 2022] PUT /identity/v3/projects/124df214b4854640b710007503ba2a33/users/9c38ae3bc3384cdba9654d5ebb362a79/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 20:00:42.125112 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9de8dc-37c0-4885-a4e8-26afaba44c8a None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:42.127708 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9de8dc-37c0-4885-a4e8-26afaba44c8a None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:42.127708 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9de8dc-37c0-4885-a4e8-26afaba44c8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:42.151703 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-ba9de8dc-37c0-4885-a4e8-26afaba44c8a None None] MFA Rules not processed for user `9c38ae3bc3384cdba9654d5ebb362a79`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 20:00:42.152900 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-ba9de8dc-37c0-4885-a4e8-26afaba44c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:00:42.214689 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1415/2814] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 20:00:42 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 20:00:42.219076 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ca3a21-27ae-442b-afd6-3b4ee14bedeb admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:42.279376 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ca3a21-27ae-442b-afd6-3b4ee14bedeb admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:42.284648 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9208d2d9-889e-4aaa-9d5e-efde90a349a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:42.285248 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208d2d9-889e-4aaa-9d5e-efde90a349a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:42.285535 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208d2d9-889e-4aaa-9d5e-efde90a349a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:42.285846 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208d2d9-889e-4aaa-9d5e-efde90a349a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:42.286542 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9208d2d9-889e-4aaa-9d5e-efde90a349a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:42.288584 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9208d2d9-889e-4aaa-9d5e-efde90a349a4 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:42.383245 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1400/2815] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 20:00:42 2022] POST /identity/v3/projects => generated 401 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 02 20:00:42.387166 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9de8dc-37c0-4885-a4e8-26afaba44c8a None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:42.398380 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9de8dc-37c0-4885-a4e8-26afaba44c8a None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:42.452387 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea213ac5-1465-41f1-a532-33baa75beb7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:42.453086 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea213ac5-1465-41f1-a532-33baa75beb7e admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:42.453378 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea213ac5-1465-41f1-a532-33baa75beb7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:42.453661 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea213ac5-1465-41f1-a532-33baa75beb7e admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:42.454351 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea213ac5-1465-41f1-a532-33baa75beb7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:42.456855 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea213ac5-1465-41f1-a532-33baa75beb7e admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:42.546833 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1416/2816] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 20:00:42 2022] POST /identity/v3/users => generated 307 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 20:00:42.550805 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9208d2d9-889e-4aaa-9d5e-efde90a349a4 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:42.561022 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9208d2d9-889e-4aaa-9d5e-efde90a349a4 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:42.566137 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ff7c92-c1b4-44e5-a363-0304e6a45a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:42.566828 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ff7c92-c1b4-44e5-a363-0304e6a45a6c admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:42.567115 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ff7c92-c1b4-44e5-a363-0304e6a45a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:42.567395 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ff7c92-c1b4-44e5-a363-0304e6a45a6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:42.568121 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ff7c92-c1b4-44e5-a363-0304e6a45a6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:42.569707 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ff7c92-c1b4-44e5-a363-0304e6a45a6c admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:42.580667 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1401/2817] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 20:00:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 20:00:42.584683 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea213ac5-1465-41f1-a532-33baa75beb7e admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:42.594488 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea213ac5-1465-41f1-a532-33baa75beb7e admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:42.599375 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538b5685-ffbc-4f78-884f-5af09ba783ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:42.600245 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b5685-ffbc-4f78-884f-5af09ba783ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:42.600527 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b5685-ffbc-4f78-884f-5af09ba783ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:42.600804 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b5685-ffbc-4f78-884f-5af09ba783ac admin admin] PATH_INFO: `/v3/projects/bb0764295867404ba121396e1aaa550f/users/1b97371ebc5940e7b1b753128c769ab6/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:42.614997 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538b5685-ffbc-4f78-884f-5af09ba783ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb0764295867404ba121396e1aaa550f, user_id=1b97371ebc5940e7b1b753128c769ab6, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:42.617630 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538b5685-ffbc-4f78-884f-5af09ba783ac admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:42.684787 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1417/2818] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 20:00:42 2022] PUT /identity/v3/projects/bb0764295867404ba121396e1aaa550f/users/1b97371ebc5940e7b1b753128c769ab6/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 20:00:42.688585 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ff7c92-c1b4-44e5-a363-0304e6a45a6c admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:42.699140 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ff7c92-c1b4-44e5-a363-0304e6a45a6c admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:42.719520 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 20:00:42.755034 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe213b1a-c259-4b05-a470-77b7becb0261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:42.755692 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe213b1a-c259-4b05-a470-77b7becb0261 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:42.755977 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe213b1a-c259-4b05-a470-77b7becb0261 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:42.756253 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe213b1a-c259-4b05-a470-77b7becb0261 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:42.756970 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe213b1a-c259-4b05-a470-77b7becb0261 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:42.758541 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe213b1a-c259-4b05-a470-77b7becb0261 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:42.769252 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1402/2819] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 20:00:42 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 20:00:42.777549 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538b5685-ffbc-4f78-884f-5af09ba783ac admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:42.782993 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 20:00:42.788198 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538b5685-ffbc-4f78-884f-5af09ba783ac admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:42.792710 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961bc772-f8a1-44ea-bd2d-f096d9e13b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:42.793595 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961bc772-f8a1-44ea-bd2d-f096d9e13b7f admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:42.793886 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961bc772-f8a1-44ea-bd2d-f096d9e13b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:42.794156 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961bc772-f8a1-44ea-bd2d-f096d9e13b7f admin admin] PATH_INFO: `/v3/projects/bb0764295867404ba121396e1aaa550f/users/1b97371ebc5940e7b1b753128c769ab6/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:42.796142 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961bc772-f8a1-44ea-bd2d-f096d9e13b7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb0764295867404ba121396e1aaa550f, user_id=1b97371ebc5940e7b1b753128c769ab6, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:42.798816 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961bc772-f8a1-44ea-bd2d-f096d9e13b7f admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:42.808587 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1418/2820] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 20:00:42 2022] PUT /identity/v3/projects/bb0764295867404ba121396e1aaa550f/users/1b97371ebc5940e7b1b753128c769ab6/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 20:00:42.818886 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e813b1-fb6a-4652-8073-43352695036c None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:42.819184 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e813b1-fb6a-4652-8073-43352695036c None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:42.819468 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e813b1-fb6a-4652-8073-43352695036c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:42.846427 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-a9e813b1-fb6a-4652-8073-43352695036c None None] MFA Rules not processed for user `1b97371ebc5940e7b1b753128c769ab6`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 20:00:42.847925 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-a9e813b1-fb6a-4652-8073-43352695036c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:00:42.915447 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1403/2821] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 20:00:42 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 20:00:42.926707 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961bc772-f8a1-44ea-bd2d-f096d9e13b7f admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:42.928587 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-961bc772-f8a1-44ea-bd2d-f096d9e13b7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:00:42.985734 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961bc772-f8a1-44ea-bd2d-f096d9e13b7f admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:42.989914 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6e801d-70f7-4915-a6d0-be43b0ada376 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:42.990604 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e801d-70f7-4915-a6d0-be43b0ada376 service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:42.990930 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e801d-70f7-4915-a6d0-be43b0ada376 service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:42.991205 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e801d-70f7-4915-a6d0-be43b0ada376 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:43.000083 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6e801d-70f7-4915-a6d0-be43b0ada376 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:43.001507 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6e801d-70f7-4915-a6d0-be43b0ada376 service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:43.011305 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1419/2822] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 20:00:42 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 20:00:43.206055 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e813b1-fb6a-4652-8073-43352695036c None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:43.215865 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e813b1-fb6a-4652-8073-43352695036c None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:43.219496 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406f76e5-4108-4f6f-82b4-fbe4bc85c30a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:43.220136 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406f76e5-4108-4f6f-82b4-fbe4bc85c30a service cinder] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:43.220425 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406f76e5-4108-4f6f-82b4-fbe4bc85c30a service cinder] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:43.220706 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406f76e5-4108-4f6f-82b4-fbe4bc85c30a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:43.278176 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406f76e5-4108-4f6f-82b4-fbe4bc85c30a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:43.279650 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406f76e5-4108-4f6f-82b4-fbe4bc85c30a service cinder] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:43.289109 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1404/2823] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 20:00:43 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 20:00:48.200424 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6e801d-70f7-4915-a6d0-be43b0ada376 service cinder] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:48.261755 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6e801d-70f7-4915-a6d0-be43b0ada376 service cinder] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:48.266223 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b62b0-6240-4b27-8cc9-28430c3272b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:48.267125 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b62b0-6240-4b27-8cc9-28430c3272b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:48.267446 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b62b0-6240-4b27-8cc9-28430c3272b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:48.267724 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b62b0-6240-4b27-8cc9-28430c3272b7 admin admin] PATH_INFO: `/v3/users/9c38ae3bc3384cdba9654d5ebb362a79` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:48.268901 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915b62b0-6240-4b27-8cc9-28430c3272b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c38ae3bc3384cdba9654d5ebb362a79)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:48.270896 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915b62b0-6240-4b27-8cc9-28430c3272b7 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:48.526289 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-915b62b0-6240-4b27-8cc9-28430c3272b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c38ae3bc3384cdba9654d5ebb362a79'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:48.533679 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-915b62b0-6240-4b27-8cc9-28430c3272b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c38ae3bc3384cdba9654d5ebb362a79'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:48.592898 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-915b62b0-6240-4b27-8cc9-28430c3272b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c38ae3bc3384cdba9654d5ebb362a79'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:48.593537 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-915b62b0-6240-4b27-8cc9-28430c3272b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c38ae3bc3384cdba9654d5ebb362a79'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:48.594202 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-915b62b0-6240-4b27-8cc9-28430c3272b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c38ae3bc3384cdba9654d5ebb362a79'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:48.613014 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1420/2824] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 20:00:48 2022] DELETE /identity/v3/users/9c38ae3bc3384cdba9654d5ebb362a79 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 20:00:48.617644 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef9234-d761-4b88-a76b-1024abfe0f6a None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:48.618153 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef9234-d761-4b88-a76b-1024abfe0f6a None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:48.618252 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef9234-d761-4b88-a76b-1024abfe0f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:48.635245 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-21ef9234-d761-4b88-a76b-1024abfe0f6a None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 20:00:48.636434 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-21ef9234-d761-4b88-a76b-1024abfe0f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:00:48.675769 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=100 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 20:00:48.693376 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1405/2825] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 20:00:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 20:00:48.702932 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b62b0-6240-4b27-8cc9-28430c3272b7 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:48.704341 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-915b62b0-6240-4b27-8cc9-28430c3272b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:00:48.717865 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 20:00:48.761550 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b62b0-6240-4b27-8cc9-28430c3272b7 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:48.765118 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cebcd7-dd6a-49d1-a3d0-b790871b8d9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:48.768440 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cebcd7-dd6a-49d1-a3d0-b790871b8d9c service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:48.768440 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cebcd7-dd6a-49d1-a3d0-b790871b8d9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:48.768440 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cebcd7-dd6a-49d1-a3d0-b790871b8d9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:48.775726 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cebcd7-dd6a-49d1-a3d0-b790871b8d9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:48.777223 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cebcd7-dd6a-49d1-a3d0-b790871b8d9c service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:48.787102 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1421/2826] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 20:00:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 20:00:49.194023 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ef9234-d761-4b88-a76b-1024abfe0f6a None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:49.198927 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-21ef9234-d761-4b88-a76b-1024abfe0f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:00:49.209690 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ef9234-d761-4b88-a76b-1024abfe0f6a None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:49.214430 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad4685a-5d9c-4aef-9d15-1f7fe40172d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:49.215250 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad4685a-5d9c-4aef-9d15-1f7fe40172d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:49.215680 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad4685a-5d9c-4aef-9d15-1f7fe40172d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:49.215955 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad4685a-5d9c-4aef-9d15-1f7fe40172d2 admin admin] PATH_INFO: `/v3/projects/124df214b4854640b710007503ba2a33` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:49.217132 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad4685a-5d9c-4aef-9d15-1f7fe40172d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=124df214b4854640b710007503ba2a33)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:49.219078 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad4685a-5d9c-4aef-9d15-1f7fe40172d2 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:49.324275 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-7ad4685a-5d9c-4aef-9d15-1f7fe40172d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '124df214b4854640b710007503ba2a33', 'request_id': 'req-7ad4685a-5d9c-4aef-9d15-1f7fe40172d2'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:49.409681 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-7ad4685a-5d9c-4aef-9d15-1f7fe40172d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '124df214b4854640b710007503ba2a33', 'request_id': 'req-7ad4685a-5d9c-4aef-9d15-1f7fe40172d2'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:49.415411 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-7ad4685a-5d9c-4aef-9d15-1f7fe40172d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '124df214b4854640b710007503ba2a33', 'request_id': 'req-7ad4685a-5d9c-4aef-9d15-1f7fe40172d2'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:49.426411 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-7ad4685a-5d9c-4aef-9d15-1f7fe40172d2 admin admin] The token cache is being invalidate because project 124df214b4854640b710007503ba2a33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 20:00:49.426704 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-7ad4685a-5d9c-4aef-9d15-1f7fe40172d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:49.427374 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-7ad4685a-5d9c-4aef-9d15-1f7fe40172d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:49.428915 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1406/2827] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 20:00:49 2022] DELETE /identity/v3/projects/124df214b4854640b710007503ba2a33 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 20:00:49.432879 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cebcd7-dd6a-49d1-a3d0-b790871b8d9c service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:49.442956 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cebcd7-dd6a-49d1-a3d0-b790871b8d9c service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:49.498513 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3f8e82-9b24-4a07-b0aa-e647575bf277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:49.499427 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3f8e82-9b24-4a07-b0aa-e647575bf277 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:49.499713 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3f8e82-9b24-4a07-b0aa-e647575bf277 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:49.499993 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3f8e82-9b24-4a07-b0aa-e647575bf277 admin admin] PATH_INFO: `/v3/users/1b97371ebc5940e7b1b753128c769ab6` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:49.501180 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3f8e82-9b24-4a07-b0aa-e647575bf277 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b97371ebc5940e7b1b753128c769ab6)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:49.503710 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3f8e82-9b24-4a07-b0aa-e647575bf277 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:49.697589 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-db3f8e82-9b24-4a07-b0aa-e647575bf277 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b97371ebc5940e7b1b753128c769ab6'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:49.704258 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-db3f8e82-9b24-4a07-b0aa-e647575bf277 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b97371ebc5940e7b1b753128c769ab6'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:49.769782 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-db3f8e82-9b24-4a07-b0aa-e647575bf277 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b97371ebc5940e7b1b753128c769ab6'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:49.770413 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-db3f8e82-9b24-4a07-b0aa-e647575bf277 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b97371ebc5940e7b1b753128c769ab6'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:49.771066 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-db3f8e82-9b24-4a07-b0aa-e647575bf277 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b97371ebc5940e7b1b753128c769ab6'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:49.788365 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1422/2828] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 20:00:49 2022] DELETE /identity/v3/users/1b97371ebc5940e7b1b753128c769ab6 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 20:00:50.193011 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad4685a-5d9c-4aef-9d15-1f7fe40172d2 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:50.194450 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-7ad4685a-5d9c-4aef-9d15-1f7fe40172d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:00:50.207984 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad4685a-5d9c-4aef-9d15-1f7fe40172d2 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:50.265895 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f208c9f4-5d8a-4ecf-ae30-8b1ffd2783d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:50.266678 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f208c9f4-5d8a-4ecf-ae30-8b1ffd2783d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:50.267044 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f208c9f4-5d8a-4ecf-ae30-8b1ffd2783d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:50.267328 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f208c9f4-5d8a-4ecf-ae30-8b1ffd2783d4 admin admin] PATH_INFO: `/v3/projects/bb0764295867404ba121396e1aaa550f` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:50.268514 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f208c9f4-5d8a-4ecf-ae30-8b1ffd2783d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb0764295867404ba121396e1aaa550f)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:50.270412 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f208c9f4-5d8a-4ecf-ae30-8b1ffd2783d4 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:50.415267 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-f208c9f4-5d8a-4ecf-ae30-8b1ffd2783d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb0764295867404ba121396e1aaa550f', 'request_id': 'req-f208c9f4-5d8a-4ecf-ae30-8b1ffd2783d4'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:50.489651 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-f208c9f4-5d8a-4ecf-ae30-8b1ffd2783d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb0764295867404ba121396e1aaa550f', 'request_id': 'req-f208c9f4-5d8a-4ecf-ae30-8b1ffd2783d4'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:50.495561 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-f208c9f4-5d8a-4ecf-ae30-8b1ffd2783d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb0764295867404ba121396e1aaa550f', 'request_id': 'req-f208c9f4-5d8a-4ecf-ae30-8b1ffd2783d4'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:50.503283 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 20:00:50.507746 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-f208c9f4-5d8a-4ecf-ae30-8b1ffd2783d4 admin admin] The token cache is being invalidate because project bb0764295867404ba121396e1aaa550f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 20:00:50.508006 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-f208c9f4-5d8a-4ecf-ae30-8b1ffd2783d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:50.508628 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-f208c9f4-5d8a-4ecf-ae30-8b1ffd2783d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:50.510177 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1407/2829] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 20:00:50 2022] DELETE /identity/v3/projects/bb0764295867404ba121396e1aaa550f => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 20:00:50.515233 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86270bc2-16c4-4669-8216-2bcb370a16c0 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:50.515704 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86270bc2-16c4-4669-8216-2bcb370a16c0 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:50.515801 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86270bc2-16c4-4669-8216-2bcb370a16c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:50.533923 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-86270bc2-16c4-4669-8216-2bcb370a16c0 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 20:00:50.534196 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-86270bc2-16c4-4669-8216-2bcb370a16c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:00:50.562908 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 20:00:50.593072 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1423/2830] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 20:00:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 20:00:50.602396 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e474f-bb30-4e32-9f26-a511f1cfaee0 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:50.602932 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e474f-bb30-4e32-9f26-a511f1cfaee0 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:50.603209 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e474f-bb30-4e32-9f26-a511f1cfaee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:50.621672 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-e18e474f-bb30-4e32-9f26-a511f1cfaee0 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 20:00:50.622908 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-e18e474f-bb30-4e32-9f26-a511f1cfaee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:00:50.631829 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1408/2831] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 20:00:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 20:00:50.635545 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86270bc2-16c4-4669-8216-2bcb370a16c0 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:50.645212 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86270bc2-16c4-4669-8216-2bcb370a16c0 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:50.649928 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2047b6a-f3b1-46b2-b6fd-b89c2e2da48c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:50.650578 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2047b6a-f3b1-46b2-b6fd-b89c2e2da48c admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:50.650896 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2047b6a-f3b1-46b2-b6fd-b89c2e2da48c admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:50.651174 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2047b6a-f3b1-46b2-b6fd-b89c2e2da48c admin admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:50.651934 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2047b6a-f3b1-46b2-b6fd-b89c2e2da48c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 20:00:50.652240 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2047b6a-f3b1-46b2-b6fd-b89c2e2da48c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:50.653827 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2047b6a-f3b1-46b2-b6fd-b89c2e2da48c admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:50.665768 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1424/2832] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 2 20:00:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 20:00:50.679929 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18e474f-bb30-4e32-9f26-a511f1cfaee0 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:50.689577 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18e474f-bb30-4e32-9f26-a511f1cfaee0 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:50.694287 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f34148-6e0b-4af5-afc6-e3058ade7781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:50.694961 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f34148-6e0b-4af5-afc6-e3058ade7781 admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:50.695252 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f34148-6e0b-4af5-afc6-e3058ade7781 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:50.695534 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f34148-6e0b-4af5-afc6-e3058ade7781 admin admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:50.696340 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f34148-6e0b-4af5-afc6-e3058ade7781 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:50.698236 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f34148-6e0b-4af5-afc6-e3058ade7781 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:50.776449 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1409/2833] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 20:00:50 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 02 20:00:50.780514 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2047b6a-f3b1-46b2-b6fd-b89c2e2da48c admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:50.791556 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2047b6a-f3b1-46b2-b6fd-b89c2e2da48c admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:50.847490 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c711f2-825e-4848-bbbd-66c19b43b6e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:50.848137 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c711f2-825e-4848-bbbd-66c19b43b6e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:50.848424 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c711f2-825e-4848-bbbd-66c19b43b6e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:50.848731 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c711f2-825e-4848-bbbd-66c19b43b6e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:50.849411 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c711f2-825e-4848-bbbd-66c19b43b6e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:50.851420 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c711f2-825e-4848-bbbd-66c19b43b6e7 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:50.924420 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1425/2834] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 20:00:50 2022] POST /identity/v3/users => generated 293 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 20:00:50.928439 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f34148-6e0b-4af5-afc6-e3058ade7781 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:50.938610 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f34148-6e0b-4af5-afc6-e3058ade7781 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:50.943139 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be6f3d6-4026-4c38-93e3-e2ae9a67269b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:50.943841 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be6f3d6-4026-4c38-93e3-e2ae9a67269b admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:50.944124 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be6f3d6-4026-4c38-93e3-e2ae9a67269b admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:50.944399 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be6f3d6-4026-4c38-93e3-e2ae9a67269b admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:50.945120 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be6f3d6-4026-4c38-93e3-e2ae9a67269b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:50.946706 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be6f3d6-4026-4c38-93e3-e2ae9a67269b admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:50.956963 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1410/2835] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 20:00:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 02 20:00:50.960729 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c711f2-825e-4848-bbbd-66c19b43b6e7 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:50.971177 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c711f2-825e-4848-bbbd-66c19b43b6e7 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:50.976113 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09fde7b-fb0b-4e33-a3f6-69529f4bd521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:50.976792 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09fde7b-fb0b-4e33-a3f6-69529f4bd521 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:50.977078 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09fde7b-fb0b-4e33-a3f6-69529f4bd521 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:50.977356 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09fde7b-fb0b-4e33-a3f6-69529f4bd521 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:50.978081 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09fde7b-fb0b-4e33-a3f6-69529f4bd521 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:50.979699 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09fde7b-fb0b-4e33-a3f6-69529f4bd521 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:50.990019 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1426/2836] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 20:00:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 20:00:50.993834 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be6f3d6-4026-4c38-93e3-e2ae9a67269b admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:51.003405 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be6f3d6-4026-4c38-93e3-e2ae9a67269b admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:51.007886 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17e369a-dfbb-40f8-a336-e862bf6e2c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:51.008741 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17e369a-dfbb-40f8-a336-e862bf6e2c3f admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:51.009026 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17e369a-dfbb-40f8-a336-e862bf6e2c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:51.009323 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17e369a-dfbb-40f8-a336-e862bf6e2c3f admin admin] PATH_INFO: `/v3/projects/1647438de3e3439dbef69ed8aa1337af/users/1b98f2c3c6944092ace06553ecffc90f/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:51.023262 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17e369a-dfbb-40f8-a336-e862bf6e2c3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1647438de3e3439dbef69ed8aa1337af, user_id=1b98f2c3c6944092ace06553ecffc90f, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:51.025843 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17e369a-dfbb-40f8-a336-e862bf6e2c3f admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:51.093623 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1411/2837] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 20:00:50 2022] PUT /identity/v3/projects/1647438de3e3439dbef69ed8aa1337af/users/1b98f2c3c6944092ace06553ecffc90f/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 20:00:51.112294 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd93c30a-d98d-439a-a869-75e19a25b9bf None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:51.112583 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd93c30a-d98d-439a-a869-75e19a25b9bf None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:51.112875 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd93c30a-d98d-439a-a869-75e19a25b9bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:51.139208 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-cd93c30a-d98d-439a-a869-75e19a25b9bf None None] MFA Rules not processed for user `1b98f2c3c6944092ace06553ecffc90f`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 20:00:51.140401 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-cd93c30a-d98d-439a-a869-75e19a25b9bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:00:51.204115 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1427/2838] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 20:00:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 20:00:51.209034 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17e369a-dfbb-40f8-a336-e862bf6e2c3f admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:51.269942 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17e369a-dfbb-40f8-a336-e862bf6e2c3f admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:51.274574 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd31102-0c2d-4af0-93c9-b8ed5b242027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:51.275264 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd31102-0c2d-4af0-93c9-b8ed5b242027 admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:51.275555 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd31102-0c2d-4af0-93c9-b8ed5b242027 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:51.275838 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd31102-0c2d-4af0-93c9-b8ed5b242027 admin admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:51.276522 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd31102-0c2d-4af0-93c9-b8ed5b242027 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:51.278401 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd31102-0c2d-4af0-93c9-b8ed5b242027 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:51.358120 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1412/2839] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 20:00:51 2022] POST /identity/v3/projects => generated 385 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 20:00:51.362199 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd93c30a-d98d-439a-a869-75e19a25b9bf None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:51.372992 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd93c30a-d98d-439a-a869-75e19a25b9bf None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:51.398993 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 20:00:51.429288 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52ee609-9e4a-4c3c-80c7-98bc7ca3c93d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:51.429962 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52ee609-9e4a-4c3c-80c7-98bc7ca3c93d admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:51.430249 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52ee609-9e4a-4c3c-80c7-98bc7ca3c93d admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:51.430528 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52ee609-9e4a-4c3c-80c7-98bc7ca3c93d admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:51.431253 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52ee609-9e4a-4c3c-80c7-98bc7ca3c93d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:51.433218 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52ee609-9e4a-4c3c-80c7-98bc7ca3c93d admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:51.541251 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1428/2840] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 20:00:51 2022] POST /identity/v3/users => generated 299 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 20:00:51.545100 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd31102-0c2d-4af0-93c9-b8ed5b242027 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:51.555194 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd31102-0c2d-4af0-93c9-b8ed5b242027 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:51.560078 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d632632f-3953-401d-986f-770f520e9ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:51.560493 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d632632f-3953-401d-986f-770f520e9ed5 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:51.560782 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d632632f-3953-401d-986f-770f520e9ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:51.561059 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d632632f-3953-401d-986f-770f520e9ed5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:51.561778 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d632632f-3953-401d-986f-770f520e9ed5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:51.563399 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d632632f-3953-401d-986f-770f520e9ed5 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:51.574135 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1413/2841] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 20:00:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 20:00:51.578085 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52ee609-9e4a-4c3c-80c7-98bc7ca3c93d admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:51.588712 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52ee609-9e4a-4c3c-80c7-98bc7ca3c93d admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:51.593362 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785e3f45-bb31-449d-8b65-b70fbea810e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:51.594237 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e3f45-bb31-449d-8b65-b70fbea810e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:51.594524 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e3f45-bb31-449d-8b65-b70fbea810e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:51.594840 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e3f45-bb31-449d-8b65-b70fbea810e3 admin admin] PATH_INFO: `/v3/projects/a68a4af56da64a1682bae6523c933009/users/50dc5f628dcf44e1b776e5c49ba1fe35/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:51.610173 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785e3f45-bb31-449d-8b65-b70fbea810e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a68a4af56da64a1682bae6523c933009, user_id=50dc5f628dcf44e1b776e5c49ba1fe35, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:51.612845 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785e3f45-bb31-449d-8b65-b70fbea810e3 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:51.688082 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1429/2842] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 20:00:51 2022] PUT /identity/v3/projects/a68a4af56da64a1682bae6523c933009/users/50dc5f628dcf44e1b776e5c49ba1fe35/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 20:00:51.691937 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d632632f-3953-401d-986f-770f520e9ed5 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:51.701843 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d632632f-3953-401d-986f-770f520e9ed5 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:51.757229 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae355879-7d0e-46ab-b5ce-c4b381eb47d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:51.760801 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae355879-7d0e-46ab-b5ce-c4b381eb47d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:51.760801 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae355879-7d0e-46ab-b5ce-c4b381eb47d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:51.760801 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae355879-7d0e-46ab-b5ce-c4b381eb47d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:51.760801 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae355879-7d0e-46ab-b5ce-c4b381eb47d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:51.760801 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae355879-7d0e-46ab-b5ce-c4b381eb47d0 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:51.770222 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1414/2843] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 20:00:51 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 20:00:51.774510 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785e3f45-bb31-449d-8b65-b70fbea810e3 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:51.785262 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785e3f45-bb31-449d-8b65-b70fbea810e3 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:51.789881 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904db3ad-5361-482f-ac16-84db6b053ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:51.790796 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904db3ad-5361-482f-ac16-84db6b053ca9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:51.791085 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904db3ad-5361-482f-ac16-84db6b053ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:51.791368 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904db3ad-5361-482f-ac16-84db6b053ca9 admin admin] PATH_INFO: `/v3/projects/a68a4af56da64a1682bae6523c933009/users/50dc5f628dcf44e1b776e5c49ba1fe35/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:51.793353 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904db3ad-5361-482f-ac16-84db6b053ca9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a68a4af56da64a1682bae6523c933009, user_id=50dc5f628dcf44e1b776e5c49ba1fe35, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:51.796031 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904db3ad-5361-482f-ac16-84db6b053ca9 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:51.807471 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1430/2844] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 20:00:51 2022] PUT /identity/v3/projects/a68a4af56da64a1682bae6523c933009/users/50dc5f628dcf44e1b776e5c49ba1fe35/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 20:00:51.817443 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7b5e8f-e896-4a16-a7ea-6b259e791453 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:51.817735 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7b5e8f-e896-4a16-a7ea-6b259e791453 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:51.818200 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7b5e8f-e896-4a16-a7ea-6b259e791453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:51.824801 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 20:00:51.843734 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-7f7b5e8f-e896-4a16-a7ea-6b259e791453 None None] MFA Rules not processed for user `50dc5f628dcf44e1b776e5c49ba1fe35`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 20:00:51.856345 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-7f7b5e8f-e896-4a16-a7ea-6b259e791453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:00:51.912171 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1415/2845] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 20:00:51 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 20:00:51.923048 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904db3ad-5361-482f-ac16-84db6b053ca9 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:51.924735 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-904db3ad-5361-482f-ac16-84db6b053ca9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:00:51.982411 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904db3ad-5361-482f-ac16-84db6b053ca9 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:51.986195 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecb8f60-5bb2-48a8-bc5e-faeae296eec0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:51.986863 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb8f60-5bb2-48a8-bc5e-faeae296eec0 service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:51.987176 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb8f60-5bb2-48a8-bc5e-faeae296eec0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:51.987456 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb8f60-5bb2-48a8-bc5e-faeae296eec0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:51.996713 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecb8f60-5bb2-48a8-bc5e-faeae296eec0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:51.998155 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecb8f60-5bb2-48a8-bc5e-faeae296eec0 service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:52.007494 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1431/2846] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 20:00:51 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 20:00:53.075195 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7b5e8f-e896-4a16-a7ea-6b259e791453 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:53.139158 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7b5e8f-e896-4a16-a7ea-6b259e791453 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:53.143766 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2f4875-5f35-4213-8d60-3a4fe5487afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:53.144631 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2f4875-5f35-4213-8d60-3a4fe5487afc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:53.144912 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2f4875-5f35-4213-8d60-3a4fe5487afc admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:53.145207 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2f4875-5f35-4213-8d60-3a4fe5487afc admin admin] PATH_INFO: `/v3/users/1b98f2c3c6944092ace06553ecffc90f` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:53.146380 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2f4875-5f35-4213-8d60-3a4fe5487afc admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b98f2c3c6944092ace06553ecffc90f)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:53.148293 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2f4875-5f35-4213-8d60-3a4fe5487afc admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:53.365004 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-4a2f4875-5f35-4213-8d60-3a4fe5487afc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b98f2c3c6944092ace06553ecffc90f'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:53.371731 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-4a2f4875-5f35-4213-8d60-3a4fe5487afc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b98f2c3c6944092ace06553ecffc90f'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:53.372466 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-4a2f4875-5f35-4213-8d60-3a4fe5487afc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b98f2c3c6944092ace06553ecffc90f'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:53.443569 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-4a2f4875-5f35-4213-8d60-3a4fe5487afc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b98f2c3c6944092ace06553ecffc90f'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:53.444219 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-4a2f4875-5f35-4213-8d60-3a4fe5487afc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b98f2c3c6944092ace06553ecffc90f'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:53.461393 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1416/2847] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 20:00:53 2022] DELETE /identity/v3/users/1b98f2c3c6944092ace06553ecffc90f => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 20:00:53.465935 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e74c66-732d-4d11-b41d-dfeadfeb1c7d None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:53.466432 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e74c66-732d-4d11-b41d-dfeadfeb1c7d None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:53.466531 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e74c66-732d-4d11-b41d-dfeadfeb1c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:53.484005 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-36e74c66-732d-4d11-b41d-dfeadfeb1c7d None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 20:00:53.485182 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-36e74c66-732d-4d11-b41d-dfeadfeb1c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:00:53.541824 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1432/2848] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 20:00:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 20:00:53.551752 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2f4875-5f35-4213-8d60-3a4fe5487afc admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:53.553259 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-4a2f4875-5f35-4213-8d60-3a4fe5487afc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:00:53.564757 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 20:00:53.610762 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2f4875-5f35-4213-8d60-3a4fe5487afc admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:53.614976 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3521e57-c62c-49f7-9245-099ad77c4559 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:53.620986 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3521e57-c62c-49f7-9245-099ad77c4559 service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:53.620986 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3521e57-c62c-49f7-9245-099ad77c4559 service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:53.620986 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3521e57-c62c-49f7-9245-099ad77c4559 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:53.625177 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3521e57-c62c-49f7-9245-099ad77c4559 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:53.627240 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3521e57-c62c-49f7-9245-099ad77c4559 service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:53.637303 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1417/2849] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 20:00:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 20:00:53.988303 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e74c66-732d-4d11-b41d-dfeadfeb1c7d None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:53.989733 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-36e74c66-732d-4d11-b41d-dfeadfeb1c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:00:54.001733 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e74c66-732d-4d11-b41d-dfeadfeb1c7d None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:54.006634 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca78d92-7499-47d3-9fd5-dd1cebee0950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:54.007545 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca78d92-7499-47d3-9fd5-dd1cebee0950 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:54.007833 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca78d92-7499-47d3-9fd5-dd1cebee0950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:54.008113 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca78d92-7499-47d3-9fd5-dd1cebee0950 admin admin] PATH_INFO: `/v3/projects/1647438de3e3439dbef69ed8aa1337af` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:54.009296 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca78d92-7499-47d3-9fd5-dd1cebee0950 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1647438de3e3439dbef69ed8aa1337af)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:54.011258 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca78d92-7499-47d3-9fd5-dd1cebee0950 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:54.033827 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 20:00:54.124668 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-eca78d92-7499-47d3-9fd5-dd1cebee0950 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1647438de3e3439dbef69ed8aa1337af', 'request_id': 'req-eca78d92-7499-47d3-9fd5-dd1cebee0950'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:54.180828 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-eca78d92-7499-47d3-9fd5-dd1cebee0950 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1647438de3e3439dbef69ed8aa1337af', 'request_id': 'req-eca78d92-7499-47d3-9fd5-dd1cebee0950'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:54.186424 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-eca78d92-7499-47d3-9fd5-dd1cebee0950 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1647438de3e3439dbef69ed8aa1337af', 'request_id': 'req-eca78d92-7499-47d3-9fd5-dd1cebee0950'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:54.197603 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-eca78d92-7499-47d3-9fd5-dd1cebee0950 admin admin] The token cache is being invalidate because project 1647438de3e3439dbef69ed8aa1337af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 20:00:54.197896 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-eca78d92-7499-47d3-9fd5-dd1cebee0950 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:54.198570 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-eca78d92-7499-47d3-9fd5-dd1cebee0950 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:54.200206 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1433/2850] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 20:00:53 2022] DELETE /identity/v3/projects/1647438de3e3439dbef69ed8aa1337af => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 20:00:54.204682 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3521e57-c62c-49f7-9245-099ad77c4559 service neutron] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:54.214239 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3521e57-c62c-49f7-9245-099ad77c4559 service neutron] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:54.269220 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d85590b-6143-45af-bcdd-01c1f9adbee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:54.270088 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d85590b-6143-45af-bcdd-01c1f9adbee1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:54.270366 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d85590b-6143-45af-bcdd-01c1f9adbee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:54.270639 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d85590b-6143-45af-bcdd-01c1f9adbee1 admin admin] PATH_INFO: `/v3/users/50dc5f628dcf44e1b776e5c49ba1fe35` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:54.271840 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d85590b-6143-45af-bcdd-01c1f9adbee1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50dc5f628dcf44e1b776e5c49ba1fe35)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:54.273730 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d85590b-6143-45af-bcdd-01c1f9adbee1 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:54.455773 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-7d85590b-6143-45af-bcdd-01c1f9adbee1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50dc5f628dcf44e1b776e5c49ba1fe35'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:54.462405 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-7d85590b-6143-45af-bcdd-01c1f9adbee1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50dc5f628dcf44e1b776e5c49ba1fe35'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:54.463069 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-7d85590b-6143-45af-bcdd-01c1f9adbee1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50dc5f628dcf44e1b776e5c49ba1fe35'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:54.527661 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-7d85590b-6143-45af-bcdd-01c1f9adbee1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50dc5f628dcf44e1b776e5c49ba1fe35'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:54.528274 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-7d85590b-6143-45af-bcdd-01c1f9adbee1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50dc5f628dcf44e1b776e5c49ba1fe35'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:54.545771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1418/2851] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 20:00:54 2022] DELETE /identity/v3/users/50dc5f628dcf44e1b776e5c49ba1fe35 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 20:00:54.947786 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca78d92-7499-47d3-9fd5-dd1cebee0950 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:54.949240 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-eca78d92-7499-47d3-9fd5-dd1cebee0950 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:00:54.966842 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca78d92-7499-47d3-9fd5-dd1cebee0950 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:55.023966 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ca6242-d71a-4fd8-85d6-1ddb8cf528e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:55.024765 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca6242-d71a-4fd8-85d6-1ddb8cf528e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:55.025055 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca6242-d71a-4fd8-85d6-1ddb8cf528e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:55.025329 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca6242-d71a-4fd8-85d6-1ddb8cf528e5 admin admin] PATH_INFO: `/v3/projects/a68a4af56da64a1682bae6523c933009` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:55.026510 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ca6242-d71a-4fd8-85d6-1ddb8cf528e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a68a4af56da64a1682bae6523c933009)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:55.028456 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ca6242-d71a-4fd8-85d6-1ddb8cf528e5 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:55.128776 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-56ca6242-d71a-4fd8-85d6-1ddb8cf528e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a68a4af56da64a1682bae6523c933009', 'request_id': 'req-56ca6242-d71a-4fd8-85d6-1ddb8cf528e5'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:55.209051 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-56ca6242-d71a-4fd8-85d6-1ddb8cf528e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a68a4af56da64a1682bae6523c933009', 'request_id': 'req-56ca6242-d71a-4fd8-85d6-1ddb8cf528e5'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:55.214689 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-56ca6242-d71a-4fd8-85d6-1ddb8cf528e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a68a4af56da64a1682bae6523c933009', 'request_id': 'req-56ca6242-d71a-4fd8-85d6-1ddb8cf528e5'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:55.226050 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-56ca6242-d71a-4fd8-85d6-1ddb8cf528e5 admin admin] The token cache is being invalidate because project a68a4af56da64a1682bae6523c933009 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 20:00:55.226343 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-56ca6242-d71a-4fd8-85d6-1ddb8cf528e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:55.227001 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-56ca6242-d71a-4fd8-85d6-1ddb8cf528e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:00:55.228561 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1434/2852] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 20:00:54 2022] DELETE /identity/v3/projects/a68a4af56da64a1682bae6523c933009 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 20:00:55.233493 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e069c84-9020-4394-8412-913475cea24c None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:55.233939 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e069c84-9020-4394-8412-913475cea24c None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:55.234101 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e069c84-9020-4394-8412-913475cea24c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:55.253293 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-9e069c84-9020-4394-8412-913475cea24c None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 20:00:55.254481 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-9e069c84-9020-4394-8412-913475cea24c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:00:55.279774 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 20:00:55.312907 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1419/2853] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 20:00:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 20:00:55.321777 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36101889-33bb-4278-8450-4008ca552374 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:55.322033 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36101889-33bb-4278-8450-4008ca552374 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:55.322316 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36101889-33bb-4278-8450-4008ca552374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:55.340411 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-36101889-33bb-4278-8450-4008ca552374 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 20:00:55.341630 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-36101889-33bb-4278-8450-4008ca552374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:00:55.350328 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1435/2854] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 20:00:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 20:00:55.354507 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e069c84-9020-4394-8412-913475cea24c None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:55.364648 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e069c84-9020-4394-8412-913475cea24c None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:55.369586 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14005371-b714-4869-a1cf-5e39750c2b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:55.370503 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14005371-b714-4869-a1cf-5e39750c2b8f admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:55.370826 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14005371-b714-4869-a1cf-5e39750c2b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:55.371111 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14005371-b714-4869-a1cf-5e39750c2b8f admin admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:55.371843 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14005371-b714-4869-a1cf-5e39750c2b8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 20:00:55.372152 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14005371-b714-4869-a1cf-5e39750c2b8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:55.373747 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14005371-b714-4869-a1cf-5e39750c2b8f admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:55.385041 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1420/2855] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 2 20:00:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 20:00:55.389269 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36101889-33bb-4278-8450-4008ca552374 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:55.398357 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36101889-33bb-4278-8450-4008ca552374 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:55.403009 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b3ebc9-2941-4542-b79e-4d253645c299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:55.403636 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3ebc9-2941-4542-b79e-4d253645c299 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:55.403920 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3ebc9-2941-4542-b79e-4d253645c299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:55.404196 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3ebc9-2941-4542-b79e-4d253645c299 admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:55.404877 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b3ebc9-2941-4542-b79e-4d253645c299 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:55.406787 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b3ebc9-2941-4542-b79e-4d253645c299 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:55.490477 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1436/2856] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 20:00:55 2022] POST /identity/v3/projects => generated 385 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 20:00:55.494544 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14005371-b714-4869-a1cf-5e39750c2b8f admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:55.504489 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14005371-b714-4869-a1cf-5e39750c2b8f admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:55.560787 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10739f8a-964a-4ba7-aaea-d74c0d1e9de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:55.561389 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10739f8a-964a-4ba7-aaea-d74c0d1e9de6 admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:55.561679 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10739f8a-964a-4ba7-aaea-d74c0d1e9de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:55.561959 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10739f8a-964a-4ba7-aaea-d74c0d1e9de6 admin admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:55.562641 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10739f8a-964a-4ba7-aaea-d74c0d1e9de6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:55.564584 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10739f8a-964a-4ba7-aaea-d74c0d1e9de6 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:55.643903 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1421/2857] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 20:00:55 2022] POST /identity/v3/users => generated 293 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 20:00:55.648049 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b3ebc9-2941-4542-b79e-4d253645c299 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:55.658921 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b3ebc9-2941-4542-b79e-4d253645c299 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:55.664018 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378671f3-d8da-48df-a110-86ff0e6f2e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:55.670283 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378671f3-d8da-48df-a110-86ff0e6f2e50 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:55.670283 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378671f3-d8da-48df-a110-86ff0e6f2e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:55.670283 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378671f3-d8da-48df-a110-86ff0e6f2e50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:55.670283 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378671f3-d8da-48df-a110-86ff0e6f2e50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:55.670283 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378671f3-d8da-48df-a110-86ff0e6f2e50 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:55.670283 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 20:00:55.678327 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1437/2858] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 20:00:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 20:00:55.682268 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10739f8a-964a-4ba7-aaea-d74c0d1e9de6 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:55.692039 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10739f8a-964a-4ba7-aaea-d74c0d1e9de6 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:55.696714 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be252b9-bd7b-4c57-8fdf-2b0f51ba49c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:55.697363 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be252b9-bd7b-4c57-8fdf-2b0f51ba49c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:55.698965 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be252b9-bd7b-4c57-8fdf-2b0f51ba49c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:55.698965 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be252b9-bd7b-4c57-8fdf-2b0f51ba49c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:55.698965 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be252b9-bd7b-4c57-8fdf-2b0f51ba49c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:55.700295 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be252b9-bd7b-4c57-8fdf-2b0f51ba49c5 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:55.710653 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1422/2859] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 20:00:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 20:00:55.714779 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378671f3-d8da-48df-a110-86ff0e6f2e50 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:55.725572 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378671f3-d8da-48df-a110-86ff0e6f2e50 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:55.730140 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d32d58-fe74-42ae-9b97-a624dee3d917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:55.731041 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d32d58-fe74-42ae-9b97-a624dee3d917 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:55.733383 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d32d58-fe74-42ae-9b97-a624dee3d917 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:55.733383 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d32d58-fe74-42ae-9b97-a624dee3d917 admin admin] PATH_INFO: `/v3/projects/4e011964dd614c588758dea4dbd00f68/users/5dbb5a3cf45e4c7581eb2c3db4dbd33f/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:55.746238 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d32d58-fe74-42ae-9b97-a624dee3d917 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e011964dd614c588758dea4dbd00f68, user_id=5dbb5a3cf45e4c7581eb2c3db4dbd33f, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:55.748949 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d32d58-fe74-42ae-9b97-a624dee3d917 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:55.854850 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1438/2860] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 20:00:55 2022] PUT /identity/v3/projects/4e011964dd614c588758dea4dbd00f68/users/5dbb5a3cf45e4c7581eb2c3db4dbd33f/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 20:00:55.864989 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd35dd01-2e31-4d9f-be49-df205244b37b None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:55.865415 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd35dd01-2e31-4d9f-be49-df205244b37b None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:55.865705 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd35dd01-2e31-4d9f-be49-df205244b37b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:55.892085 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-fd35dd01-2e31-4d9f-be49-df205244b37b None None] MFA Rules not processed for user `5dbb5a3cf45e4c7581eb2c3db4dbd33f`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 20:00:55.893294 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-fd35dd01-2e31-4d9f-be49-df205244b37b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:00:55.956464 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1423/2861] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 20:00:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 20:00:55.967017 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d32d58-fe74-42ae-9b97-a624dee3d917 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:00:55.973712 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-70d32d58-fe74-42ae-9b97-a624dee3d917 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:00:56.025494 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d32d58-fe74-42ae-9b97-a624dee3d917 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:00:56.029231 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a86352-18d1-4566-8369-c9b215f26854 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:00:56.029892 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a86352-18d1-4566-8369-c9b215f26854 service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:00:56.030169 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a86352-18d1-4566-8369-c9b215f26854 service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:00:56.030449 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a86352-18d1-4566-8369-c9b215f26854 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:00:56.038925 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a86352-18d1-4566-8369-c9b215f26854 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:00:56.040372 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a86352-18d1-4566-8369-c9b215f26854 service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:00:56.049115 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1439/2862] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 20:00:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 20:01:05.840163 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd35dd01-2e31-4d9f-be49-df205244b37b None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:01:05.870130 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 20:01:05.901453 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd35dd01-2e31-4d9f-be49-df205244b37b None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:01:05.907059 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215430f6-0b4e-40a6-8361-009f2f5e8c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:01:05.907926 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215430f6-0b4e-40a6-8361-009f2f5e8c15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:05.908211 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215430f6-0b4e-40a6-8361-009f2f5e8c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:05.908488 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215430f6-0b4e-40a6-8361-009f2f5e8c15 admin admin] PATH_INFO: `/v3/users/5dbb5a3cf45e4c7581eb2c3db4dbd33f` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:05.909681 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215430f6-0b4e-40a6-8361-009f2f5e8c15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5dbb5a3cf45e4c7581eb2c3db4dbd33f)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:01:05.911671 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215430f6-0b4e-40a6-8361-009f2f5e8c15 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:01:06.044251 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-215430f6-0b4e-40a6-8361-009f2f5e8c15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5dbb5a3cf45e4c7581eb2c3db4dbd33f'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:01:06.045079 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=37,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 20:01:06.051819 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-215430f6-0b4e-40a6-8361-009f2f5e8c15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5dbb5a3cf45e4c7581eb2c3db4dbd33f'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:01:06.052500 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-215430f6-0b4e-40a6-8361-009f2f5e8c15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5dbb5a3cf45e4c7581eb2c3db4dbd33f'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:01:06.093404 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-215430f6-0b4e-40a6-8361-009f2f5e8c15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5dbb5a3cf45e4c7581eb2c3db4dbd33f'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:01:06.094570 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-215430f6-0b4e-40a6-8361-009f2f5e8c15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5dbb5a3cf45e4c7581eb2c3db4dbd33f'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:01:06.112627 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1424/2863] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 20:01:05 2022] DELETE /identity/v3/users/5dbb5a3cf45e4c7581eb2c3db4dbd33f => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 20:01:06.117140 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97173f67-a7ed-48b8-b1f4-bb2b7f5b78b4 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:06.117435 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97173f67-a7ed-48b8-b1f4-bb2b7f5b78b4 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:06.117719 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97173f67-a7ed-48b8-b1f4-bb2b7f5b78b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:06.134592 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-97173f67-a7ed-48b8-b1f4-bb2b7f5b78b4 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 20:01:06.135829 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-97173f67-a7ed-48b8-b1f4-bb2b7f5b78b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:01:06.194014 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1440/2864] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 20:01:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 20:01:06.204023 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215430f6-0b4e-40a6-8361-009f2f5e8c15 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:01:06.205444 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-215430f6-0b4e-40a6-8361-009f2f5e8c15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:01:06.261332 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215430f6-0b4e-40a6-8361-009f2f5e8c15 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:01:06.264941 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d7cb0f-5fbe-41f0-b2cc-fded9281aa13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:01:06.266092 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d7cb0f-5fbe-41f0-b2cc-fded9281aa13 service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:06.266376 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d7cb0f-5fbe-41f0-b2cc-fded9281aa13 service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:06.266654 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d7cb0f-5fbe-41f0-b2cc-fded9281aa13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:06.275784 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d7cb0f-5fbe-41f0-b2cc-fded9281aa13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:01:06.277205 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d7cb0f-5fbe-41f0-b2cc-fded9281aa13 service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:01:06.287165 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1425/2865] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 20:01:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 20:01:06.683302 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97173f67-a7ed-48b8-b1f4-bb2b7f5b78b4 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:01:06.684739 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-97173f67-a7ed-48b8-b1f4-bb2b7f5b78b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:01:06.697119 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97173f67-a7ed-48b8-b1f4-bb2b7f5b78b4 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:01:06.701404 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48942ce-d78b-4d91-b5dd-83c55dbb06ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:01:06.702307 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48942ce-d78b-4d91-b5dd-83c55dbb06ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:06.702596 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48942ce-d78b-4d91-b5dd-83c55dbb06ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:06.702910 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48942ce-d78b-4d91-b5dd-83c55dbb06ab admin admin] PATH_INFO: `/v3/projects/4e011964dd614c588758dea4dbd00f68` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:06.704110 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48942ce-d78b-4d91-b5dd-83c55dbb06ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e011964dd614c588758dea4dbd00f68)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:01:06.706031 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48942ce-d78b-4d91-b5dd-83c55dbb06ab admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:01:06.819072 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f48942ce-d78b-4d91-b5dd-83c55dbb06ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e011964dd614c588758dea4dbd00f68', 'request_id': 'req-f48942ce-d78b-4d91-b5dd-83c55dbb06ab'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:01:06.887589 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f48942ce-d78b-4d91-b5dd-83c55dbb06ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e011964dd614c588758dea4dbd00f68', 'request_id': 'req-f48942ce-d78b-4d91-b5dd-83c55dbb06ab'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:01:06.893877 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f48942ce-d78b-4d91-b5dd-83c55dbb06ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e011964dd614c588758dea4dbd00f68', 'request_id': 'req-f48942ce-d78b-4d91-b5dd-83c55dbb06ab'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:01:06.904866 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f48942ce-d78b-4d91-b5dd-83c55dbb06ab admin admin] The token cache is being invalidate because project 4e011964dd614c588758dea4dbd00f68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 20:01:06.905152 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f48942ce-d78b-4d91-b5dd-83c55dbb06ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:01:06.905771 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f48942ce-d78b-4d91-b5dd-83c55dbb06ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:01:06.907932 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1441/2866] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 20:01:06 2022] DELETE /identity/v3/projects/4e011964dd614c588758dea4dbd00f68 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 20:01:06.912744 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a537275-f60b-4834-9fd2-57c9175d7004 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:06.913216 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a537275-f60b-4834-9fd2-57c9175d7004 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:06.913306 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a537275-f60b-4834-9fd2-57c9175d7004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:06.930379 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-6a537275-f60b-4834-9fd2-57c9175d7004 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 20:01:06.931541 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-6a537275-f60b-4834-9fd2-57c9175d7004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:01:06.975237 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 20:01:06.990068 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1426/2867] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 20:01:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 20:01:06.998551 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf7eb3-0d43-4de5-bc6b-cb3d230fd0bf None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:06.998887 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf7eb3-0d43-4de5-bc6b-cb3d230fd0bf None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:06.999175 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf7eb3-0d43-4de5-bc6b-cb3d230fd0bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:07.018577 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-04cf7eb3-0d43-4de5-bc6b-cb3d230fd0bf None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 20:01:07.023324 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-04cf7eb3-0d43-4de5-bc6b-cb3d230fd0bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:01:07.027989 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1442/2868] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 20:01:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 20:01:07.032599 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a537275-f60b-4834-9fd2-57c9175d7004 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:01:07.042161 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a537275-f60b-4834-9fd2-57c9175d7004 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:01:07.047411 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea00e14-5960-4861-b5f6-cc1e96273212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:01:07.054620 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea00e14-5960-4861-b5f6-cc1e96273212 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:07.054620 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea00e14-5960-4861-b5f6-cc1e96273212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:07.054620 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea00e14-5960-4861-b5f6-cc1e96273212 admin admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:07.054620 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea00e14-5960-4861-b5f6-cc1e96273212 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 20:01:07.054620 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea00e14-5960-4861-b5f6-cc1e96273212 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:01:07.054620 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea00e14-5960-4861-b5f6-cc1e96273212 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:01:07.063035 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1427/2869] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 2 20:01:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 02 20:01:07.067149 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cf7eb3-0d43-4de5-bc6b-cb3d230fd0bf None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:01:07.076152 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cf7eb3-0d43-4de5-bc6b-cb3d230fd0bf None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:01:07.081249 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae2c5bb-6d75-4f36-bfa8-1860efd07ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:01:07.081886 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae2c5bb-6d75-4f36-bfa8-1860efd07ca8 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:07.082165 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae2c5bb-6d75-4f36-bfa8-1860efd07ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:07.082439 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae2c5bb-6d75-4f36-bfa8-1860efd07ca8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:07.083163 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae2c5bb-6d75-4f36-bfa8-1860efd07ca8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:01:07.085046 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae2c5bb-6d75-4f36-bfa8-1860efd07ca8 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:01:07.168313 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1443/2870] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 20:01:07 2022] POST /identity/v3/projects => generated 375 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 20:01:07.172325 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea00e14-5960-4861-b5f6-cc1e96273212 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:01:07.182290 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea00e14-5960-4861-b5f6-cc1e96273212 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:01:07.236581 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c308b076-2cb0-469f-99ff-5ea82c88b980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:01:07.237247 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c308b076-2cb0-469f-99ff-5ea82c88b980 admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:07.237537 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c308b076-2cb0-469f-99ff-5ea82c88b980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:07.237819 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c308b076-2cb0-469f-99ff-5ea82c88b980 admin admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:07.238505 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c308b076-2cb0-469f-99ff-5ea82c88b980 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:01:07.240454 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c308b076-2cb0-469f-99ff-5ea82c88b980 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:01:07.333234 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1428/2871] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 20:01:07 2022] POST /identity/v3/users => generated 288 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 20:01:07.337104 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae2c5bb-6d75-4f36-bfa8-1860efd07ca8 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:01:07.347667 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae2c5bb-6d75-4f36-bfa8-1860efd07ca8 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:01:07.352294 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b501bdb1-0571-4836-a602-5451f29a4fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:01:07.352949 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b501bdb1-0571-4836-a602-5451f29a4fb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:07.353232 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b501bdb1-0571-4836-a602-5451f29a4fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:07.353593 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b501bdb1-0571-4836-a602-5451f29a4fb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:07.354235 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b501bdb1-0571-4836-a602-5451f29a4fb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:01:07.355899 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b501bdb1-0571-4836-a602-5451f29a4fb8 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:01:07.366059 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1444/2872] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 20:01:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 20:01:07.369762 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c308b076-2cb0-469f-99ff-5ea82c88b980 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:01:07.379911 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c308b076-2cb0-469f-99ff-5ea82c88b980 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:01:07.384501 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e64c85a-85b4-4298-b3f1-8edfa6ecac78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:01:07.385148 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e64c85a-85b4-4298-b3f1-8edfa6ecac78 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:07.385432 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e64c85a-85b4-4298-b3f1-8edfa6ecac78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:07.385710 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e64c85a-85b4-4298-b3f1-8edfa6ecac78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:07.386429 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e64c85a-85b4-4298-b3f1-8edfa6ecac78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:01:07.388046 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e64c85a-85b4-4298-b3f1-8edfa6ecac78 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:01:07.398244 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1429/2873] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 20:01:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 20:01:07.402160 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b501bdb1-0571-4836-a602-5451f29a4fb8 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:01:07.411992 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b501bdb1-0571-4836-a602-5451f29a4fb8 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:01:07.416630 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b816d205-2c98-4125-8d5b-027401877808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:01:07.417507 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b816d205-2c98-4125-8d5b-027401877808 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:07.417791 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b816d205-2c98-4125-8d5b-027401877808 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:07.418069 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b816d205-2c98-4125-8d5b-027401877808 admin admin] PATH_INFO: `/v3/projects/dffa2a06e0d742f181fc3e2200e749dd/users/8f3d1ade405b455cb7bd6af0e8accc3e/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:07.432576 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b816d205-2c98-4125-8d5b-027401877808 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dffa2a06e0d742f181fc3e2200e749dd, user_id=8f3d1ade405b455cb7bd6af0e8accc3e, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:01:07.435225 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b816d205-2c98-4125-8d5b-027401877808 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:01:07.504836 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1445/2874] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 20:01:07 2022] PUT /identity/v3/projects/dffa2a06e0d742f181fc3e2200e749dd/users/8f3d1ade405b455cb7bd6af0e8accc3e/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 20:01:07.514945 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40ba64a-f49f-4b78-be3e-4ae40555a756 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:07.515264 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40ba64a-f49f-4b78-be3e-4ae40555a756 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:07.516187 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40ba64a-f49f-4b78-be3e-4ae40555a756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:07.541869 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-c40ba64a-f49f-4b78-be3e-4ae40555a756 None None] MFA Rules not processed for user `8f3d1ade405b455cb7bd6af0e8accc3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 20:01:07.543109 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-c40ba64a-f49f-4b78-be3e-4ae40555a756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:01:07.606211 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1430/2875] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 20:01:07 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 20:01:07.610829 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b816d205-2c98-4125-8d5b-027401877808 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:01:07.631958 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 20:01:07.672962 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b816d205-2c98-4125-8d5b-027401877808 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:01:07.677538 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00ad54f-6524-4a50-864e-f86de29d9cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:01:07.678188 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00ad54f-6524-4a50-864e-f86de29d9cd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:07.678472 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00ad54f-6524-4a50-864e-f86de29d9cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:07.678766 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00ad54f-6524-4a50-864e-f86de29d9cd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:07.679535 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00ad54f-6524-4a50-864e-f86de29d9cd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:01:07.681504 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00ad54f-6524-4a50-864e-f86de29d9cd0 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:01:07.761325 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1446/2876] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 20:01:07 2022] POST /identity/v3/projects => generated 375 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 02 20:01:07.765644 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40ba64a-f49f-4b78-be3e-4ae40555a756 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:01:07.776196 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40ba64a-f49f-4b78-be3e-4ae40555a756 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:01:07.817432 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 20:01:07.831492 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447afef1-0bed-449e-961b-744812cfa2ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:01:07.832253 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447afef1-0bed-449e-961b-744812cfa2ae admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:07.832554 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447afef1-0bed-449e-961b-744812cfa2ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:07.832833 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447afef1-0bed-449e-961b-744812cfa2ae admin admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:07.833521 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447afef1-0bed-449e-961b-744812cfa2ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:01:07.835448 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447afef1-0bed-449e-961b-744812cfa2ae admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:01:07.938549 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1431/2877] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 20:01:07 2022] POST /identity/v3/users => generated 294 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 20:01:07.942215 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00ad54f-6524-4a50-864e-f86de29d9cd0 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:01:07.952477 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00ad54f-6524-4a50-864e-f86de29d9cd0 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:01:07.957102 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0108cb6d-7fa9-423d-9246-98aff6cac91f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:01:07.957753 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0108cb6d-7fa9-423d-9246-98aff6cac91f admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:07.958037 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0108cb6d-7fa9-423d-9246-98aff6cac91f admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:07.958313 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0108cb6d-7fa9-423d-9246-98aff6cac91f admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:07.959109 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0108cb6d-7fa9-423d-9246-98aff6cac91f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:01:07.964111 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0108cb6d-7fa9-423d-9246-98aff6cac91f admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:01:07.971550 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1447/2878] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 20:01:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 20:01:07.975568 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447afef1-0bed-449e-961b-744812cfa2ae admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:01:07.985821 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447afef1-0bed-449e-961b-744812cfa2ae admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:01:07.990664 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f868ce49-27b1-4677-b7e1-b0cd230d34ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:01:07.991573 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f868ce49-27b1-4677-b7e1-b0cd230d34ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:07.991859 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f868ce49-27b1-4677-b7e1-b0cd230d34ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:07.992137 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f868ce49-27b1-4677-b7e1-b0cd230d34ed admin admin] PATH_INFO: `/v3/projects/c5614a4060b84e1da0de61f78fd73cc5/users/c73d895c1b364999990b089f5c943093/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:08.007291 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f868ce49-27b1-4677-b7e1-b0cd230d34ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5614a4060b84e1da0de61f78fd73cc5, user_id=c73d895c1b364999990b089f5c943093, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:01:08.009944 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f868ce49-27b1-4677-b7e1-b0cd230d34ed admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:01:08.070138 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1432/2879] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 20:01:07 2022] PUT /identity/v3/projects/c5614a4060b84e1da0de61f78fd73cc5/users/c73d895c1b364999990b089f5c943093/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 20:01:08.073910 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0108cb6d-7fa9-423d-9246-98aff6cac91f admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:01:08.083663 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0108cb6d-7fa9-423d-9246-98aff6cac91f admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:01:08.141007 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0cc422-f1de-4a28-af7b-7d21da7d2ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:01:08.141677 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0cc422-f1de-4a28-af7b-7d21da7d2ac4 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:08.141967 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0cc422-f1de-4a28-af7b-7d21da7d2ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:08.142263 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0cc422-f1de-4a28-af7b-7d21da7d2ac4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:08.143014 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0cc422-f1de-4a28-af7b-7d21da7d2ac4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:01:08.144632 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0cc422-f1de-4a28-af7b-7d21da7d2ac4 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:01:08.154217 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1448/2880] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 20:01:08 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 20:01:08.158763 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f868ce49-27b1-4677-b7e1-b0cd230d34ed admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:01:08.168771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f868ce49-27b1-4677-b7e1-b0cd230d34ed admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:01:08.174030 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac7a807-9919-4140-8e26-a67d1245d8cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:01:08.174948 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac7a807-9919-4140-8e26-a67d1245d8cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:08.175235 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac7a807-9919-4140-8e26-a67d1245d8cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:08.175512 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac7a807-9919-4140-8e26-a67d1245d8cd admin admin] PATH_INFO: `/v3/projects/c5614a4060b84e1da0de61f78fd73cc5/users/c73d895c1b364999990b089f5c943093/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:08.177479 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac7a807-9919-4140-8e26-a67d1245d8cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5614a4060b84e1da0de61f78fd73cc5, user_id=c73d895c1b364999990b089f5c943093, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:01:08.180153 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac7a807-9919-4140-8e26-a67d1245d8cd admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:01:08.190400 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1433/2881] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 20:01:08 2022] PUT /identity/v3/projects/c5614a4060b84e1da0de61f78fd73cc5/users/c73d895c1b364999990b089f5c943093/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 20:01:08.200135 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e5e1ae-c391-478a-aeff-1a87f78d5a2e None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:08.200426 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e5e1ae-c391-478a-aeff-1a87f78d5a2e None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:08.200708 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e5e1ae-c391-478a-aeff-1a87f78d5a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:08.226702 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-a7e5e1ae-c391-478a-aeff-1a87f78d5a2e None None] MFA Rules not processed for user `c73d895c1b364999990b089f5c943093`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 20:01:08.227937 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-a7e5e1ae-c391-478a-aeff-1a87f78d5a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:01:08.293593 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1449/2882] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 20:01:08 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 20:01:08.304914 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac7a807-9919-4140-8e26-a67d1245d8cd admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:01:08.306599 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-0ac7a807-9919-4140-8e26-a67d1245d8cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:01:08.363924 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac7a807-9919-4140-8e26-a67d1245d8cd admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:01:08.367523 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75f8d0a-cfa5-4b06-95de-819a1d8d7811 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:01:08.368169 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f8d0a-cfa5-4b06-95de-819a1d8d7811 service cinder] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:08.368457 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f8d0a-cfa5-4b06-95de-819a1d8d7811 service cinder] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:08.368738 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f8d0a-cfa5-4b06-95de-819a1d8d7811 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:08.423472 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75f8d0a-cfa5-4b06-95de-819a1d8d7811 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:01:08.424907 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75f8d0a-cfa5-4b06-95de-819a1d8d7811 service cinder] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:01:08.433545 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1434/2883] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 20:01:08 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 20:01:11.447932 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e5e1ae-c391-478a-aeff-1a87f78d5a2e None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:01:11.465147 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 20:01:11.509198 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e5e1ae-c391-478a-aeff-1a87f78d5a2e None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:01:11.513792 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a4e13a-d6a9-458a-99b4-8d4e08ed60d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:01:11.514699 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a4e13a-d6a9-458a-99b4-8d4e08ed60d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:11.515031 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a4e13a-d6a9-458a-99b4-8d4e08ed60d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:11.515295 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a4e13a-d6a9-458a-99b4-8d4e08ed60d5 admin admin] PATH_INFO: `/v3/users/8f3d1ade405b455cb7bd6af0e8accc3e` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:11.516961 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a4e13a-d6a9-458a-99b4-8d4e08ed60d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f3d1ade405b455cb7bd6af0e8accc3e)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:01:11.518925 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a4e13a-d6a9-458a-99b4-8d4e08ed60d5 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:01:11.688733 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-b8a4e13a-d6a9-458a-99b4-8d4e08ed60d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f3d1ade405b455cb7bd6af0e8accc3e'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:01:11.695436 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-b8a4e13a-d6a9-458a-99b4-8d4e08ed60d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f3d1ade405b455cb7bd6af0e8accc3e'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:01:11.784491 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-b8a4e13a-d6a9-458a-99b4-8d4e08ed60d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f3d1ade405b455cb7bd6af0e8accc3e'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:01:11.785117 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-b8a4e13a-d6a9-458a-99b4-8d4e08ed60d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f3d1ade405b455cb7bd6af0e8accc3e'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:01:11.785732 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-b8a4e13a-d6a9-458a-99b4-8d4e08ed60d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f3d1ade405b455cb7bd6af0e8accc3e'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:01:11.804220 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1450/2884] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 20:01:11 2022] DELETE /identity/v3/users/8f3d1ade405b455cb7bd6af0e8accc3e => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 20:01:11.808777 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aee5c3a-7b75-4c60-9460-483216a3eb31 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:11.809248 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aee5c3a-7b75-4c60-9460-483216a3eb31 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:11.809338 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aee5c3a-7b75-4c60-9460-483216a3eb31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:11.827716 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-0aee5c3a-7b75-4c60-9460-483216a3eb31 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 20:01:11.828970 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-0aee5c3a-7b75-4c60-9460-483216a3eb31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:01:11.873564 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 20:01:11.887795 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1435/2885] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 20:01:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 20:01:11.897845 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a4e13a-d6a9-458a-99b4-8d4e08ed60d5 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:01:11.899269 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-b8a4e13a-d6a9-458a-99b4-8d4e08ed60d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:01:11.956369 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a4e13a-d6a9-458a-99b4-8d4e08ed60d5 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:01:11.959932 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c70361-8a41-494b-aca5-eb10028bcb51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:01:11.960584 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c70361-8a41-494b-aca5-eb10028bcb51 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:11.960870 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c70361-8a41-494b-aca5-eb10028bcb51 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:11.961155 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c70361-8a41-494b-aca5-eb10028bcb51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:11.970528 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c70361-8a41-494b-aca5-eb10028bcb51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:01:11.972030 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c70361-8a41-494b-aca5-eb10028bcb51 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:01:11.981396 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1451/2886] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 20:01:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 20:01:12.334587 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aee5c3a-7b75-4c60-9460-483216a3eb31 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:01:12.336063 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-0aee5c3a-7b75-4c60-9460-483216a3eb31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:01:12.348214 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aee5c3a-7b75-4c60-9460-483216a3eb31 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:01:12.352905 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea51107a-a5e3-4e4c-af61-de384d00119b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:01:12.353793 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea51107a-a5e3-4e4c-af61-de384d00119b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:12.354072 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea51107a-a5e3-4e4c-af61-de384d00119b admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:12.354341 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea51107a-a5e3-4e4c-af61-de384d00119b admin admin] PATH_INFO: `/v3/projects/dffa2a06e0d742f181fc3e2200e749dd` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:12.355734 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea51107a-a5e3-4e4c-af61-de384d00119b admin admin] RBAC: Authorizing `identity:delete_project(project_id=dffa2a06e0d742f181fc3e2200e749dd)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:01:12.357631 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea51107a-a5e3-4e4c-af61-de384d00119b admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:01:12.453715 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-ea51107a-a5e3-4e4c-af61-de384d00119b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dffa2a06e0d742f181fc3e2200e749dd', 'request_id': 'req-ea51107a-a5e3-4e4c-af61-de384d00119b'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:01:12.509765 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-ea51107a-a5e3-4e4c-af61-de384d00119b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dffa2a06e0d742f181fc3e2200e749dd', 'request_id': 'req-ea51107a-a5e3-4e4c-af61-de384d00119b'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:01:12.515247 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-ea51107a-a5e3-4e4c-af61-de384d00119b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dffa2a06e0d742f181fc3e2200e749dd', 'request_id': 'req-ea51107a-a5e3-4e4c-af61-de384d00119b'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:01:12.526099 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-ea51107a-a5e3-4e4c-af61-de384d00119b admin admin] The token cache is being invalidate because project dffa2a06e0d742f181fc3e2200e749dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 20:01:12.526365 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-ea51107a-a5e3-4e4c-af61-de384d00119b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:01:12.527013 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-ea51107a-a5e3-4e4c-af61-de384d00119b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:01:12.528538 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1436/2887] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 20:01:12 2022] DELETE /identity/v3/projects/dffa2a06e0d742f181fc3e2200e749dd => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 20:01:12.533117 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c70361-8a41-494b-aca5-eb10028bcb51 service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:01:12.548797 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c70361-8a41-494b-aca5-eb10028bcb51 service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:01:12.581404 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 20:01:12.610834 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2645f977-761b-49c3-bfbb-02625109f10c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:01:12.611739 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645f977-761b-49c3-bfbb-02625109f10c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:12.612023 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645f977-761b-49c3-bfbb-02625109f10c admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:12.612299 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645f977-761b-49c3-bfbb-02625109f10c admin admin] PATH_INFO: `/v3/users/c73d895c1b364999990b089f5c943093` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:12.613485 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2645f977-761b-49c3-bfbb-02625109f10c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c73d895c1b364999990b089f5c943093)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:01:12.615458 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2645f977-761b-49c3-bfbb-02625109f10c admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:01:12.803370 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-2645f977-761b-49c3-bfbb-02625109f10c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c73d895c1b364999990b089f5c943093'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:01:12.810369 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-2645f977-761b-49c3-bfbb-02625109f10c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c73d895c1b364999990b089f5c943093'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:01:12.887281 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-2645f977-761b-49c3-bfbb-02625109f10c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c73d895c1b364999990b089f5c943093'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:01:12.887910 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-2645f977-761b-49c3-bfbb-02625109f10c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c73d895c1b364999990b089f5c943093'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:01:12.888545 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-2645f977-761b-49c3-bfbb-02625109f10c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c73d895c1b364999990b089f5c943093'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:01:12.906371 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1452/2888] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 20:01:12 2022] DELETE /identity/v3/users/c73d895c1b364999990b089f5c943093 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 20:01:13.515843 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea51107a-a5e3-4e4c-af61-de384d00119b admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:01:13.517328 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-ea51107a-a5e3-4e4c-af61-de384d00119b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:01:13.530534 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea51107a-a5e3-4e4c-af61-de384d00119b admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:01:13.588014 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38453bd1-5106-4458-8ee8-a0185c850b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:01:13.588871 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38453bd1-5106-4458-8ee8-a0185c850b92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:13.589154 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38453bd1-5106-4458-8ee8-a0185c850b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:13.589434 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38453bd1-5106-4458-8ee8-a0185c850b92 admin admin] PATH_INFO: `/v3/projects/c5614a4060b84e1da0de61f78fd73cc5` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:13.590600 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38453bd1-5106-4458-8ee8-a0185c850b92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5614a4060b84e1da0de61f78fd73cc5)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:01:13.592516 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38453bd1-5106-4458-8ee8-a0185c850b92 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:01:13.697498 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-38453bd1-5106-4458-8ee8-a0185c850b92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5614a4060b84e1da0de61f78fd73cc5', 'request_id': 'req-38453bd1-5106-4458-8ee8-a0185c850b92'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:01:13.749484 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-38453bd1-5106-4458-8ee8-a0185c850b92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5614a4060b84e1da0de61f78fd73cc5', 'request_id': 'req-38453bd1-5106-4458-8ee8-a0185c850b92'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:01:13.755081 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-38453bd1-5106-4458-8ee8-a0185c850b92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5614a4060b84e1da0de61f78fd73cc5', 'request_id': 'req-38453bd1-5106-4458-8ee8-a0185c850b92'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:01:13.765986 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-38453bd1-5106-4458-8ee8-a0185c850b92 admin admin] The token cache is being invalidate because project c5614a4060b84e1da0de61f78fd73cc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 20:01:13.766839 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-38453bd1-5106-4458-8ee8-a0185c850b92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:01:13.767474 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-38453bd1-5106-4458-8ee8-a0185c850b92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:01:13.768974 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1437/2889] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 20:01:13 2022] DELETE /identity/v3/projects/c5614a4060b84e1da0de61f78fd73cc5 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 02 20:01:13.773800 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdca2ec-4e4d-48cf-abb2-d9eb980b8ba3 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:13.774294 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdca2ec-4e4d-48cf-abb2-d9eb980b8ba3 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:13.774391 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdca2ec-4e4d-48cf-abb2-d9eb980b8ba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:13.792091 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-0fdca2ec-4e4d-48cf-abb2-d9eb980b8ba3 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 20:01:13.793293 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-0fdca2ec-4e4d-48cf-abb2-d9eb980b8ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:01:13.851281 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1453/2890] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 20:01:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 20:01:13.859921 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b638ac13-12db-4c97-a5e3-7553ced87cef None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:13.860200 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b638ac13-12db-4c97-a5e3-7553ced87cef None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:13.860501 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b638ac13-12db-4c97-a5e3-7553ced87cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:13.867922 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 20:01:13.878668 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-b638ac13-12db-4c97-a5e3-7553ced87cef None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 20:01:13.879967 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-b638ac13-12db-4c97-a5e3-7553ced87cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:01:13.887963 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1438/2891] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 20:01:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 20:01:13.892683 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdca2ec-4e4d-48cf-abb2-d9eb980b8ba3 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:01:13.902124 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdca2ec-4e4d-48cf-abb2-d9eb980b8ba3 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:01:13.908299 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baeeb83-f1cc-43ae-8f38-1219022065eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:01:13.908958 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baeeb83-f1cc-43ae-8f38-1219022065eb admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:13.909247 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baeeb83-f1cc-43ae-8f38-1219022065eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:13.909623 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baeeb83-f1cc-43ae-8f38-1219022065eb admin admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:13.910452 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baeeb83-f1cc-43ae-8f38-1219022065eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 20:01:13.910788 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baeeb83-f1cc-43ae-8f38-1219022065eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:01:13.912395 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baeeb83-f1cc-43ae-8f38-1219022065eb admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:01:13.924657 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1454/2892] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 2 20:01:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 20:01:13.928871 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b638ac13-12db-4c97-a5e3-7553ced87cef None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:01:13.937658 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b638ac13-12db-4c97-a5e3-7553ced87cef None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:01:13.942373 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff563c23-3e6d-4b14-8018-424eb5bae9bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:01:13.943122 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff563c23-3e6d-4b14-8018-424eb5bae9bd admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:13.943406 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff563c23-3e6d-4b14-8018-424eb5bae9bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:13.943678 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff563c23-3e6d-4b14-8018-424eb5bae9bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:13.944362 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff563c23-3e6d-4b14-8018-424eb5bae9bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:01:13.946290 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff563c23-3e6d-4b14-8018-424eb5bae9bd admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:01:14.043062 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1439/2893] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 20:01:13 2022] POST /identity/v3/projects => generated 373 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 20:01:14.048588 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baeeb83-f1cc-43ae-8f38-1219022065eb admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:01:14.060022 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baeeb83-f1cc-43ae-8f38-1219022065eb admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:01:14.103850 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 20:01:14.119036 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3233625a-0ca4-4d9c-b766-0e10838552d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:01:14.119844 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3233625a-0ca4-4d9c-b766-0e10838552d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:14.120112 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3233625a-0ca4-4d9c-b766-0e10838552d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:14.120393 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3233625a-0ca4-4d9c-b766-0e10838552d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:14.121106 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3233625a-0ca4-4d9c-b766-0e10838552d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:01:14.123091 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3233625a-0ca4-4d9c-b766-0e10838552d2 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:01:14.223019 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1455/2894] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 20:01:14 2022] POST /identity/v3/users => generated 287 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 20:01:14.227203 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff563c23-3e6d-4b14-8018-424eb5bae9bd admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:01:14.237421 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff563c23-3e6d-4b14-8018-424eb5bae9bd admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:01:14.242103 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6b315d-af4e-4da9-b17e-b3cd39781e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:01:14.242793 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b315d-af4e-4da9-b17e-b3cd39781e1e admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:14.243081 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b315d-af4e-4da9-b17e-b3cd39781e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:14.243365 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b315d-af4e-4da9-b17e-b3cd39781e1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:14.244108 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6b315d-af4e-4da9-b17e-b3cd39781e1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:01:14.245684 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6b315d-af4e-4da9-b17e-b3cd39781e1e admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:01:14.256036 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1440/2895] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 20:01:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 20:01:14.259866 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3233625a-0ca4-4d9c-b766-0e10838552d2 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:01:14.270249 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3233625a-0ca4-4d9c-b766-0e10838552d2 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:01:14.275085 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bfbd2d-0dc8-4b15-8964-6e9456d57575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:01:14.275735 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bfbd2d-0dc8-4b15-8964-6e9456d57575 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:14.276015 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bfbd2d-0dc8-4b15-8964-6e9456d57575 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:14.276289 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bfbd2d-0dc8-4b15-8964-6e9456d57575 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:14.277039 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bfbd2d-0dc8-4b15-8964-6e9456d57575 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:01:14.278614 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bfbd2d-0dc8-4b15-8964-6e9456d57575 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:01:14.288808 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1456/2896] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 20:01:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 20:01:14.292835 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6b315d-af4e-4da9-b17e-b3cd39781e1e admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:01:14.302887 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6b315d-af4e-4da9-b17e-b3cd39781e1e admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:01:14.307447 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9de519-2975-4c5b-ac54-f8578b9f3dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:01:14.308316 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9de519-2975-4c5b-ac54-f8578b9f3dec admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:14.308598 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9de519-2975-4c5b-ac54-f8578b9f3dec admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:14.308871 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9de519-2975-4c5b-ac54-f8578b9f3dec admin admin] PATH_INFO: `/v3/projects/ec39efbb5b894ee7a22d72b70d1f5f7d/users/4f291c269d114ed5b9a692ba80a2a7ef/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:14.323780 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9de519-2975-4c5b-ac54-f8578b9f3dec admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec39efbb5b894ee7a22d72b70d1f5f7d, user_id=4f291c269d114ed5b9a692ba80a2a7ef, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:01:14.326527 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9de519-2975-4c5b-ac54-f8578b9f3dec admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:01:14.395778 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1441/2897] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 20:01:14 2022] PUT /identity/v3/projects/ec39efbb5b894ee7a22d72b70d1f5f7d/users/4f291c269d114ed5b9a692ba80a2a7ef/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 20:01:14.400952 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70d92d-0149-43ee-a26a-0cb1f14cf287 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:14.401249 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70d92d-0149-43ee-a26a-0cb1f14cf287 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:14.401529 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70d92d-0149-43ee-a26a-0cb1f14cf287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:14.418761 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-db70d92d-0149-43ee-a26a-0cb1f14cf287 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 20:01:14.420016 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-db70d92d-0149-43ee-a26a-0cb1f14cf287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:01:14.476765 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1457/2898] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 20:01:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 20:01:14.486720 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9de519-2975-4c5b-ac54-f8578b9f3dec admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:01:14.488439 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-6e9de519-2975-4c5b-ac54-f8578b9f3dec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:01:14.550776 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9de519-2975-4c5b-ac54-f8578b9f3dec admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:01:14.554459 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ed4342-d4a7-42a8-a2c4-2b16339f09fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:01:14.555075 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ed4342-d4a7-42a8-a2c4-2b16339f09fe service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:14.555362 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ed4342-d4a7-42a8-a2c4-2b16339f09fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:14.555638 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ed4342-d4a7-42a8-a2c4-2b16339f09fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:14.565128 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ed4342-d4a7-42a8-a2c4-2b16339f09fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:01:14.566548 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ed4342-d4a7-42a8-a2c4-2b16339f09fe service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:01:14.575792 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1442/2899] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 20:01:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 20:01:15.734484 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1080d225-83f7-44e9-b0b9-f549f61ed9fb None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:15.734484 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1080d225-83f7-44e9-b0b9-f549f61ed9fb None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:15.734484 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1080d225-83f7-44e9-b0b9-f549f61ed9fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:15.761620 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-1080d225-83f7-44e9-b0b9-f549f61ed9fb None None] MFA Rules not processed for user `4f291c269d114ed5b9a692ba80a2a7ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 20:01:15.763120 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-1080d225-83f7-44e9-b0b9-f549f61ed9fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:01:15.828923 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1458/2900] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 20:01:15 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 20:01:15.840965 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ed4342-d4a7-42a8-a2c4-2b16339f09fe service neutron] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:01:15.842377 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-c2ed4342-d4a7-42a8-a2c4-2b16339f09fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:01:15.900117 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ed4342-d4a7-42a8-a2c4-2b16339f09fe service neutron] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:01:15.903823 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42900fc1-b366-4d00-8d71-30c65d9afede service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '091860b2577f49c5af61c75803a90c0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:01:15.906348 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42900fc1-b366-4d00-8d71-30c65d9afede service cinder] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:15.906348 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42900fc1-b366-4d00-8d71-30c65d9afede service cinder] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:15.906348 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42900fc1-b366-4d00-8d71-30c65d9afede service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:15.914000 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42900fc1-b366-4d00-8d71-30c65d9afede service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:01:15.915468 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42900fc1-b366-4d00-8d71-30c65d9afede service cinder] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:01:15.924119 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1443/2901] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 20:01:15 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 20:01:17.499171 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1996221f-cf7d-4210-a7f2-eab7e6246e68 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:17.499676 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1996221f-cf7d-4210-a7f2-eab7e6246e68 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:17.499779 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1996221f-cf7d-4210-a7f2-eab7e6246e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:17.517643 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-1996221f-cf7d-4210-a7f2-eab7e6246e68 None None] MFA Rules not processed for user `4f291c269d114ed5b9a692ba80a2a7ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 20:01:17.518868 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-1996221f-cf7d-4210-a7f2-eab7e6246e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:01:17.525642 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1459/2902] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 20:01:17 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 20:01:17.748640 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42900fc1-b366-4d00-8d71-30c65d9afede service cinder] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:01:17.750125 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-42900fc1-b366-4d00-8d71-30c65d9afede service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:01:17.762019 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42900fc1-b366-4d00-8d71-30c65d9afede service cinder] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:01:17.765560 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f67713-7e35-48a6-9a05-378cf2f56660 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:01:17.766196 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f67713-7e35-48a6-9a05-378cf2f56660 service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:17.766482 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f67713-7e35-48a6-9a05-378cf2f56660 service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:17.766792 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f67713-7e35-48a6-9a05-378cf2f56660 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:17.768548 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-09f67713-7e35-48a6-9a05-378cf2f56660 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:01:17.777715 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f67713-7e35-48a6-9a05-378cf2f56660 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:01:17.779179 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f67713-7e35-48a6-9a05-378cf2f56660 service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:01:17.787693 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1444/2903] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 20:01:17 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 20:01:18.166510 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1996221f-cf7d-4210-a7f2-eab7e6246e68 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:01:18.167939 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-1996221f-cf7d-4210-a7f2-eab7e6246e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:01:18.220175 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 20:01:18.241521 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1996221f-cf7d-4210-a7f2-eab7e6246e68 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:01:18.246783 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebedf20-aa9b-4580-8fe9-93d4b7ffd127 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73ae837e99764bebbe7a6e06ac9049e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:01:18.247432 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebedf20-aa9b-4580-8fe9-93d4b7ffd127 service placement] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:18.247763 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebedf20-aa9b-4580-8fe9-93d4b7ffd127 service placement] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:18.248043 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebedf20-aa9b-4580-8fe9-93d4b7ffd127 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:18.249833 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-eebedf20-aa9b-4580-8fe9-93d4b7ffd127 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:01:18.261284 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebedf20-aa9b-4580-8fe9-93d4b7ffd127 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:01:18.262947 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebedf20-aa9b-4580-8fe9-93d4b7ffd127 service placement] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:01:18.273909 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1460/2904] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 20:01:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 20:01:23.044780 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f67713-7e35-48a6-9a05-378cf2f56660 service neutron] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:01:23.046221 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-09f67713-7e35-48a6-9a05-378cf2f56660 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:01:23.083557 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 20:01:23.121451 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f67713-7e35-48a6-9a05-378cf2f56660 service neutron] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:01:23.127784 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de037a85-15f4-4cd9-ba47-9576ffa1c729 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:01:23.128436 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de037a85-15f4-4cd9-ba47-9576ffa1c729 service nova] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:23.128720 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de037a85-15f4-4cd9-ba47-9576ffa1c729 service nova] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:23.128996 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de037a85-15f4-4cd9-ba47-9576ffa1c729 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:23.131119 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-de037a85-15f4-4cd9-ba47-9576ffa1c729 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:01:23.144767 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de037a85-15f4-4cd9-ba47-9576ffa1c729 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:01:23.146437 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de037a85-15f4-4cd9-ba47-9576ffa1c729 service nova] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:01:23.157784 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1445/2905] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 20:01:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 20:01:28.269714 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=11 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 20:01:28.661568 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebedf20-aa9b-4580-8fe9-93d4b7ffd127 service placement] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:01:28.674509 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebedf20-aa9b-4580-8fe9-93d4b7ffd127 service placement] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:01:28.680034 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69eecc0-5c01-4a17-8cf0-a28ac0a8d1f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:01:28.683572 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69eecc0-5c01-4a17-8cf0-a28ac0a8d1f4 service nova] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:28.683572 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69eecc0-5c01-4a17-8cf0-a28ac0a8d1f4 service nova] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:28.683572 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69eecc0-5c01-4a17-8cf0-a28ac0a8d1f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:28.683572 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-c69eecc0-5c01-4a17-8cf0-a28ac0a8d1f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:01:28.696814 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69eecc0-5c01-4a17-8cf0-a28ac0a8d1f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:01:28.698443 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69eecc0-5c01-4a17-8cf0-a28ac0a8d1f4 service nova] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:01:28.710987 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1461/2906] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 20:01:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 20:01:31.535614 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de037a85-15f4-4cd9-ba47-9576ffa1c729 service nova] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:01:31.547058 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de037a85-15f4-4cd9-ba47-9576ffa1c729 service nova] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:01:31.554543 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d41a335-604a-48a8-9d9c-29b35cc79de3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:01:31.556625 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41a335-604a-48a8-9d9c-29b35cc79de3 service nova] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:31.556625 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41a335-604a-48a8-9d9c-29b35cc79de3 service nova] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:31.556625 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41a335-604a-48a8-9d9c-29b35cc79de3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:31.557782 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-1d41a335-604a-48a8-9d9c-29b35cc79de3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:01:31.615507 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d41a335-604a-48a8-9d9c-29b35cc79de3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:01:31.617198 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d41a335-604a-48a8-9d9c-29b35cc79de3 service nova] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:01:31.627338 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1446/2907] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 20:01:31 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 20:01:33.313082 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69eecc0-5c01-4a17-8cf0-a28ac0a8d1f4 service nova] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:01:33.322708 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69eecc0-5c01-4a17-8cf0-a28ac0a8d1f4 service nova] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:01:33.326479 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f258dba-75fc-4b5d-b772-2cbfbb7e0abf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:01:33.327147 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f258dba-75fc-4b5d-b772-2cbfbb7e0abf service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:33.327437 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f258dba-75fc-4b5d-b772-2cbfbb7e0abf service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:33.327719 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f258dba-75fc-4b5d-b772-2cbfbb7e0abf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:33.330076 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-2f258dba-75fc-4b5d-b772-2cbfbb7e0abf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:01:33.344978 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f258dba-75fc-4b5d-b772-2cbfbb7e0abf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:01:33.346468 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f258dba-75fc-4b5d-b772-2cbfbb7e0abf service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:01:33.358858 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1462/2908] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 20:01:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 20:01:34.273051 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d41a335-604a-48a8-9d9c-29b35cc79de3 service nova] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:01:34.284031 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d41a335-604a-48a8-9d9c-29b35cc79de3 service nova] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:01:34.287663 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d0ea41-a4ec-408c-95fd-fc0ce0b56266 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:01:34.288346 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d0ea41-a4ec-408c-95fd-fc0ce0b56266 service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:34.288661 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d0ea41-a4ec-408c-95fd-fc0ce0b56266 service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:34.288986 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d0ea41-a4ec-408c-95fd-fc0ce0b56266 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:34.290859 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-d5d0ea41-a4ec-408c-95fd-fc0ce0b56266 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:01:34.303278 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d0ea41-a4ec-408c-95fd-fc0ce0b56266 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:01:34.304704 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d0ea41-a4ec-408c-95fd-fc0ce0b56266 service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:01:34.314304 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1447/2909] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 20:01:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 20:01:35.785309 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0c23ab-032e-4298-b0ca-cab397173e71 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:35.785793 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0c23ab-032e-4298-b0ca-cab397173e71 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:35.785870 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0c23ab-032e-4298-b0ca-cab397173e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:35.805381 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-fc0c23ab-032e-4298-b0ca-cab397173e71 None None] MFA Rules not processed for user `4f291c269d114ed5b9a692ba80a2a7ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 20:01:35.806769 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-fc0c23ab-032e-4298-b0ca-cab397173e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:01:35.813876 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1463/2910] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 20:01:35 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 20:01:36.095107 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d0ea41-a4ec-408c-95fd-fc0ce0b56266 service neutron] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:01:36.105892 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d0ea41-a4ec-408c-95fd-fc0ce0b56266 service neutron] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:01:36.109846 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181c682a-3ded-4fdd-baab-e3cc99815736 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:01:36.110510 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181c682a-3ded-4fdd-baab-e3cc99815736 service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:36.110825 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181c682a-3ded-4fdd-baab-e3cc99815736 service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:36.111109 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181c682a-3ded-4fdd-baab-e3cc99815736 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:36.121528 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-181c682a-3ded-4fdd-baab-e3cc99815736 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:01:36.132725 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181c682a-3ded-4fdd-baab-e3cc99815736 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:01:36.134437 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181c682a-3ded-4fdd-baab-e3cc99815736 service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:01:36.143956 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1448/2911] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 20:01:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 20:01:42.495612 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0c23ab-032e-4298-b0ca-cab397173e71 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:01:42.507819 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0c23ab-032e-4298-b0ca-cab397173e71 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:01:42.512852 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016e6656-80ca-4a87-8431-c5eb6583d0fd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73ae837e99764bebbe7a6e06ac9049e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:01:42.513490 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016e6656-80ca-4a87-8431-c5eb6583d0fd service placement] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:42.513772 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016e6656-80ca-4a87-8431-c5eb6583d0fd service placement] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:42.514046 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016e6656-80ca-4a87-8431-c5eb6583d0fd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:42.516316 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-016e6656-80ca-4a87-8431-c5eb6583d0fd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:01:42.527882 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016e6656-80ca-4a87-8431-c5eb6583d0fd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:01:42.529446 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016e6656-80ca-4a87-8431-c5eb6583d0fd service placement] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:01:42.539790 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1464/2912] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 20:01:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 20:01:42.626483 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181c682a-3ded-4fdd-baab-e3cc99815736 service neutron] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:01:42.627990 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-181c682a-3ded-4fdd-baab-e3cc99815736 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:01:42.643206 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181c682a-3ded-4fdd-baab-e3cc99815736 service neutron] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:01:42.647685 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d02e8a-ccee-46d0-8f84-52f042cf863c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:01:42.648555 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d02e8a-ccee-46d0-8f84-52f042cf863c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:42.648833 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d02e8a-ccee-46d0-8f84-52f042cf863c admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:42.649104 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d02e8a-ccee-46d0-8f84-52f042cf863c admin admin] PATH_INFO: `/v3/users/674e6df4247b45278d237035f31801e6` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:42.650292 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d02e8a-ccee-46d0-8f84-52f042cf863c admin admin] RBAC: Authorizing `identity:delete_user(user_id=674e6df4247b45278d237035f31801e6)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:01:42.652225 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d02e8a-ccee-46d0-8f84-52f042cf863c admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:01:42.859694 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-53d02e8a-ccee-46d0-8f84-52f042cf863c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '674e6df4247b45278d237035f31801e6'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:01:42.867133 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-53d02e8a-ccee-46d0-8f84-52f042cf863c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '674e6df4247b45278d237035f31801e6'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:01:42.867919 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-53d02e8a-ccee-46d0-8f84-52f042cf863c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '674e6df4247b45278d237035f31801e6'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:01:42.935062 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-53d02e8a-ccee-46d0-8f84-52f042cf863c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '674e6df4247b45278d237035f31801e6'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:01:42.935690 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-53d02e8a-ccee-46d0-8f84-52f042cf863c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '674e6df4247b45278d237035f31801e6'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:01:42.953717 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1449/2913] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 20:01:42 2022] DELETE /identity/v3/users/674e6df4247b45278d237035f31801e6 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 20:01:43.214837 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016e6656-80ca-4a87-8431-c5eb6583d0fd service placement] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:01:43.230483 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016e6656-80ca-4a87-8431-c5eb6583d0fd service placement] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:01:43.290513 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f3df0d-a0dc-40ed-aae3-146a1c79a751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:01:43.291436 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f3df0d-a0dc-40ed-aae3-146a1c79a751 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:43.291720 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f3df0d-a0dc-40ed-aae3-146a1c79a751 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:43.291999 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f3df0d-a0dc-40ed-aae3-146a1c79a751 admin admin] PATH_INFO: `/v3/projects/d8ca263ee3f0443a8d59b31b2c32449d` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:43.293176 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f3df0d-a0dc-40ed-aae3-146a1c79a751 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8ca263ee3f0443a8d59b31b2c32449d)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:01:43.295112 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f3df0d-a0dc-40ed-aae3-146a1c79a751 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:01:43.397861 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-98f3df0d-a0dc-40ed-aae3-146a1c79a751 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8ca263ee3f0443a8d59b31b2c32449d', 'request_id': 'req-98f3df0d-a0dc-40ed-aae3-146a1c79a751'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:01:43.466616 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-98f3df0d-a0dc-40ed-aae3-146a1c79a751 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8ca263ee3f0443a8d59b31b2c32449d', 'request_id': 'req-98f3df0d-a0dc-40ed-aae3-146a1c79a751'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:01:43.473299 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-98f3df0d-a0dc-40ed-aae3-146a1c79a751 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8ca263ee3f0443a8d59b31b2c32449d', 'request_id': 'req-98f3df0d-a0dc-40ed-aae3-146a1c79a751'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:01:43.484883 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-98f3df0d-a0dc-40ed-aae3-146a1c79a751 admin admin] The token cache is being invalidate because project d8ca263ee3f0443a8d59b31b2c32449d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 20:01:43.485165 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-98f3df0d-a0dc-40ed-aae3-146a1c79a751 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:01:43.485797 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-98f3df0d-a0dc-40ed-aae3-146a1c79a751 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:01:43.487371 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1465/2914] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 20:01:43 2022] DELETE /identity/v3/projects/d8ca263ee3f0443a8d59b31b2c32449d => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 20:01:43.491412 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d02e8a-ccee-46d0-8f84-52f042cf863c admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:01:43.492824 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-53d02e8a-ccee-46d0-8f84-52f042cf863c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:01:43.500968 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 20:01:43.506257 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d02e8a-ccee-46d0-8f84-52f042cf863c admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:01:43.562207 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66b3d3d-10e6-4668-9ad3-8c262a001a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:01:43.563154 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66b3d3d-10e6-4668-9ad3-8c262a001a9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:43.563440 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66b3d3d-10e6-4668-9ad3-8c262a001a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:43.563720 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66b3d3d-10e6-4668-9ad3-8c262a001a9e admin admin] PATH_INFO: `/v3/users/e7be4a79a5da472e83f32a112b260794` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:43.564903 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66b3d3d-10e6-4668-9ad3-8c262a001a9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7be4a79a5da472e83f32a112b260794)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:01:43.566880 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66b3d3d-10e6-4668-9ad3-8c262a001a9e admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:01:43.766300 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-d66b3d3d-10e6-4668-9ad3-8c262a001a9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7be4a79a5da472e83f32a112b260794'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:01:43.772912 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-d66b3d3d-10e6-4668-9ad3-8c262a001a9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7be4a79a5da472e83f32a112b260794'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:01:43.773577 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-d66b3d3d-10e6-4668-9ad3-8c262a001a9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7be4a79a5da472e83f32a112b260794'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:01:43.820561 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-d66b3d3d-10e6-4668-9ad3-8c262a001a9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7be4a79a5da472e83f32a112b260794'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:01:43.821192 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-d66b3d3d-10e6-4668-9ad3-8c262a001a9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7be4a79a5da472e83f32a112b260794'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:01:43.839427 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1450/2915] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 20:01:43 2022] DELETE /identity/v3/users/e7be4a79a5da472e83f32a112b260794 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 20:01:44.153249 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f3df0d-a0dc-40ed-aae3-146a1c79a751 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:01:44.154738 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-98f3df0d-a0dc-40ed-aae3-146a1c79a751 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:01:44.167588 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f3df0d-a0dc-40ed-aae3-146a1c79a751 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:01:44.211164 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 20:01:44.225798 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7926d5d-420d-4639-8366-531e62cc0e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:01:44.226669 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7926d5d-420d-4639-8366-531e62cc0e21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:44.226986 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7926d5d-420d-4639-8366-531e62cc0e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:44.227264 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7926d5d-420d-4639-8366-531e62cc0e21 admin admin] PATH_INFO: `/v3/projects/d4211c56449c4a17af374b99dbf47777` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:44.228437 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7926d5d-420d-4639-8366-531e62cc0e21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4211c56449c4a17af374b99dbf47777)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:01:44.230344 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7926d5d-420d-4639-8366-531e62cc0e21 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:01:44.367960 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-d7926d5d-420d-4639-8366-531e62cc0e21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4211c56449c4a17af374b99dbf47777', 'request_id': 'req-d7926d5d-420d-4639-8366-531e62cc0e21'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:01:44.453803 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-d7926d5d-420d-4639-8366-531e62cc0e21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4211c56449c4a17af374b99dbf47777', 'request_id': 'req-d7926d5d-420d-4639-8366-531e62cc0e21'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:01:44.460156 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-d7926d5d-420d-4639-8366-531e62cc0e21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4211c56449c4a17af374b99dbf47777', 'request_id': 'req-d7926d5d-420d-4639-8366-531e62cc0e21'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:01:44.471722 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-d7926d5d-420d-4639-8366-531e62cc0e21 admin admin] The token cache is being invalidate because project d4211c56449c4a17af374b99dbf47777 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 20:01:44.472010 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-d7926d5d-420d-4639-8366-531e62cc0e21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:01:44.474983 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-d7926d5d-420d-4639-8366-531e62cc0e21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:01:44.474983 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1466/2916] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 20:01:44 2022] DELETE /identity/v3/projects/d4211c56449c4a17af374b99dbf47777 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 20:01:44.479367 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba58dd6c-2353-4727-ae23-01b403256517 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:44.479668 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba58dd6c-2353-4727-ae23-01b403256517 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:44.479956 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba58dd6c-2353-4727-ae23-01b403256517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:44.500206 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-ba58dd6c-2353-4727-ae23-01b403256517 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 20:01:44.501434 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-ba58dd6c-2353-4727-ae23-01b403256517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:01:44.561500 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1451/2917] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 20:01:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 20:01:44.570129 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc828b6-1b6f-4eda-869b-3d7eacd59e35 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:44.570448 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc828b6-1b6f-4eda-869b-3d7eacd59e35 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:44.570731 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc828b6-1b6f-4eda-869b-3d7eacd59e35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:44.589598 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-3fc828b6-1b6f-4eda-869b-3d7eacd59e35 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 20:01:44.591128 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-3fc828b6-1b6f-4eda-869b-3d7eacd59e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:01:44.599607 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1467/2918] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 20:01:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 20:01:44.609807 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba58dd6c-2353-4727-ae23-01b403256517 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:01:44.619796 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba58dd6c-2353-4727-ae23-01b403256517 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:01:44.624944 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0db877-d870-4dd8-a3dc-4a25ef5d0fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:01:44.625602 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0db877-d870-4dd8-a3dc-4a25ef5d0fc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:44.625886 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0db877-d870-4dd8-a3dc-4a25ef5d0fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:44.626164 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0db877-d870-4dd8-a3dc-4a25ef5d0fc0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:44.629822 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0db877-d870-4dd8-a3dc-4a25ef5d0fc0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80492) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 02 20:01:44.629822 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0db877-d870-4dd8-a3dc-4a25ef5d0fc0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:01:44.629822 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0db877-d870-4dd8-a3dc-4a25ef5d0fc0 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:01:44.640592 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1452/2919] 10.222.0.33 () {58 vars in 1152 bytes} [Thu Jun 2 20:01:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 02 20:01:44.644830 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc828b6-1b6f-4eda-869b-3d7eacd59e35 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:01:44.653963 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc828b6-1b6f-4eda-869b-3d7eacd59e35 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:01:44.659113 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf884552-f1ff-4171-9a8d-1254551712d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:01:44.659753 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf884552-f1ff-4171-9a8d-1254551712d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:44.660036 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf884552-f1ff-4171-9a8d-1254551712d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:44.660307 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf884552-f1ff-4171-9a8d-1254551712d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:44.660990 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf884552-f1ff-4171-9a8d-1254551712d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:01:44.662899 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf884552-f1ff-4171-9a8d-1254551712d1 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:01:44.766831 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1468/2920] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 20:01:44 2022] POST /identity/v3/projects => generated 379 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 20:01:44.773749 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0db877-d870-4dd8-a3dc-4a25ef5d0fc0 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:01:44.785194 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0db877-d870-4dd8-a3dc-4a25ef5d0fc0 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:01:44.789771 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 20:01:44.839917 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4df9165-c00c-4a85-87d4-46cc7ed67fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:01:44.840587 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df9165-c00c-4a85-87d4-46cc7ed67fe9 admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:44.841140 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df9165-c00c-4a85-87d4-46cc7ed67fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:44.841419 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df9165-c00c-4a85-87d4-46cc7ed67fe9 admin admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:44.842108 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4df9165-c00c-4a85-87d4-46cc7ed67fe9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:01:44.844592 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4df9165-c00c-4a85-87d4-46cc7ed67fe9 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:01:44.932598 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1453/2921] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 20:01:44 2022] POST /identity/v3/users => generated 290 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 20:01:44.933871 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf884552-f1ff-4171-9a8d-1254551712d1 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:01:44.944935 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf884552-f1ff-4171-9a8d-1254551712d1 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:01:44.951469 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6924b6d-3428-42eb-ab97-fab7d3dfc86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:01:44.951469 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6924b6d-3428-42eb-ab97-fab7d3dfc86f admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:44.951469 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6924b6d-3428-42eb-ab97-fab7d3dfc86f admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:44.951469 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6924b6d-3428-42eb-ab97-fab7d3dfc86f admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:44.952093 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6924b6d-3428-42eb-ab97-fab7d3dfc86f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:01:44.953725 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6924b6d-3428-42eb-ab97-fab7d3dfc86f admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:01:44.967577 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1469/2922] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 20:01:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 20:01:44.969284 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4df9165-c00c-4a85-87d4-46cc7ed67fe9 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:01:44.979500 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4df9165-c00c-4a85-87d4-46cc7ed67fe9 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:01:44.984746 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f65689-62b8-44c9-91fb-1b5cd4a12af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:01:44.985414 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f65689-62b8-44c9-91fb-1b5cd4a12af8 admin admin] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:44.985700 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f65689-62b8-44c9-91fb-1b5cd4a12af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:44.985976 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f65689-62b8-44c9-91fb-1b5cd4a12af8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:44.986740 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f65689-62b8-44c9-91fb-1b5cd4a12af8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:01:44.988346 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f65689-62b8-44c9-91fb-1b5cd4a12af8 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:01:45.001343 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1454/2923] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 20:01:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 20:01:45.002518 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6924b6d-3428-42eb-ab97-fab7d3dfc86f admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:01:45.012477 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6924b6d-3428-42eb-ab97-fab7d3dfc86f admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:01:45.017126 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdbfe22-32f5-4f90-b722-4799a80afc8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:01:45.018004 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdbfe22-32f5-4f90-b722-4799a80afc8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:45.018291 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdbfe22-32f5-4f90-b722-4799a80afc8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:45.018572 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdbfe22-32f5-4f90-b722-4799a80afc8b admin admin] PATH_INFO: `/v3/projects/4a81da1e3d824c7691c755d265050732/users/3a519787e4944a78a93d48bd9e729282/roles/77d5301ed78146468d4115b512cc70b7` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:45.033187 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdbfe22-32f5-4f90-b722-4799a80afc8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a81da1e3d824c7691c755d265050732, user_id=3a519787e4944a78a93d48bd9e729282, role_id=77d5301ed78146468d4115b512cc70b7)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:01:45.035882 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdbfe22-32f5-4f90-b722-4799a80afc8b admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:01:45.100671 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1470/2924] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 20:01:45 2022] PUT /identity/v3/projects/4a81da1e3d824c7691c755d265050732/users/3a519787e4944a78a93d48bd9e729282/roles/77d5301ed78146468d4115b512cc70b7 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 02 20:01:45.105811 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173e0300-2378-4adb-8157-fa862d1a45d8 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:45.106153 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173e0300-2378-4adb-8157-fa862d1a45d8 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:45.106430 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173e0300-2378-4adb-8157-fa862d1a45d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:45.124751 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-173e0300-2378-4adb-8157-fa862d1a45d8 None None] MFA Rules not processed for user `789d0dc0871243f79a6f4f6f602c4f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 20:01:45.129001 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-173e0300-2378-4adb-8157-fa862d1a45d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:01:45.185441 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1455/2925] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 20:01:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 20:01:45.194989 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdbfe22-32f5-4f90-b722-4799a80afc8b admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:01:45.196711 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-dfdbfe22-32f5-4f90-b722-4799a80afc8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:01:45.255604 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdbfe22-32f5-4f90-b722-4799a80afc8b admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:01:45.259213 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3baa2f5-8739-4adb-bbf9-5efa2cbfd950 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:01:45.259855 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3baa2f5-8739-4adb-bbf9-5efa2cbfd950 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:45.260136 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3baa2f5-8739-4adb-bbf9-5efa2cbfd950 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:45.260408 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3baa2f5-8739-4adb-bbf9-5efa2cbfd950 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:45.270248 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3baa2f5-8739-4adb-bbf9-5efa2cbfd950 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:01:45.271734 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3baa2f5-8739-4adb-bbf9-5efa2cbfd950 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:01:45.281917 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1471/2926] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 20:01:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 20:01:50.597632 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8c05b-7057-4e17-8c86-ea83ca297c51 None None] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:50.598032 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8c05b-7057-4e17-8c86-ea83ca297c51 None None] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:50.598256 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8c05b-7057-4e17-8c86-ea83ca297c51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:50.629837 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.auth.core [None req-b1c8c05b-7057-4e17-8c86-ea83ca297c51 None None] MFA Rules not processed for user `3a519787e4944a78a93d48bd9e729282`. Rule list: `[]` (Enabled: `True`). {{(pid=80492) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 20:01:50.632445 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-b1c8c05b-7057-4e17-8c86-ea83ca297c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:01:50.707920 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1456/2927] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 20:01:50 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 20:01:50.712521 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3baa2f5-8739-4adb-bbf9-5efa2cbfd950 service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:01:50.726440 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3baa2f5-8739-4adb-bbf9-5efa2cbfd950 service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:01:50.731426 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2c3b96-ad86-4b18-8ab2-83ea691404ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:01:50.732096 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2c3b96-ad86-4b18-8ab2-83ea691404ef admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:50.732395 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2c3b96-ad86-4b18-8ab2-83ea691404ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:50.732679 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2c3b96-ad86-4b18-8ab2-83ea691404ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:50.733996 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2c3b96-ad86-4b18-8ab2-83ea691404ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:01:50.735980 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2c3b96-ad86-4b18-8ab2-83ea691404ef admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:01:50.748051 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 20:01:50.822375 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1472/2928] 10.222.0.33 () {60 vars in 1152 bytes} [Thu Jun 2 20:01:50 2022] POST /identity/v3/projects => generated 377 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 20:01:50.825997 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c8c05b-7057-4e17-8c86-ea83ca297c51 None None] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:01:50.837280 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c8c05b-7057-4e17-8c86-ea83ca297c51 None None] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:01:50.852205 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 20:01:50.897423 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a52e546-db0e-4217-af77-64ab9fb9e2e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:01:50.898165 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a52e546-db0e-4217-af77-64ab9fb9e2e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:50.898451 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a52e546-db0e-4217-af77-64ab9fb9e2e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:50.898731 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a52e546-db0e-4217-af77-64ab9fb9e2e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:50.901770 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a52e546-db0e-4217-af77-64ab9fb9e2e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:01:50.902147 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a52e546-db0e-4217-af77-64ab9fb9e2e2 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:01:51.005367 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1457/2929] 10.222.0.33 () {60 vars in 1143 bytes} [Thu Jun 2 20:01:50 2022] POST /identity/v3/users => generated 295 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 02 20:01:51.009396 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2c3b96-ad86-4b18-8ab2-83ea691404ef admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:01:51.020246 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2c3b96-ad86-4b18-8ab2-83ea691404ef admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:01:51.025227 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf1ac36-be93-4921-9fe0-350a37391245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:01:51.025985 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf1ac36-be93-4921-9fe0-350a37391245 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:51.026280 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf1ac36-be93-4921-9fe0-350a37391245 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:51.026560 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf1ac36-be93-4921-9fe0-350a37391245 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:51.027350 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf1ac36-be93-4921-9fe0-350a37391245 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:01:51.028979 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf1ac36-be93-4921-9fe0-350a37391245 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:01:51.041512 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1473/2930] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 20:01:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 20:01:51.046181 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a52e546-db0e-4217-af77-64ab9fb9e2e2 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:01:51.056935 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a52e546-db0e-4217-af77-64ab9fb9e2e2 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:01:51.061970 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d297b1c7-7b93-49a1-9554-ffe202d92bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:01:51.062914 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d297b1c7-7b93-49a1-9554-ffe202d92bda admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:51.063214 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d297b1c7-7b93-49a1-9554-ffe202d92bda admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:51.063465 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d297b1c7-7b93-49a1-9554-ffe202d92bda admin admin] PATH_INFO: `/v3/projects/d37c8d040e474fa381c945cacbfa2446/users/2bdf71883a8c4c449395efb60d4c0ad0/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:51.081837 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d297b1c7-7b93-49a1-9554-ffe202d92bda admin admin] RBAC: Authorizing `identity:create_grant(project_id=d37c8d040e474fa381c945cacbfa2446, user_id=2bdf71883a8c4c449395efb60d4c0ad0, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:01:51.095215 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d297b1c7-7b93-49a1-9554-ffe202d92bda admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:01:51.166655 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1458/2931] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 20:01:51 2022] PUT /identity/v3/projects/d37c8d040e474fa381c945cacbfa2446/users/2bdf71883a8c4c449395efb60d4c0ad0/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 20:01:51.170297 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf1ac36-be93-4921-9fe0-350a37391245 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:01:51.181790 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf1ac36-be93-4921-9fe0-350a37391245 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:01:51.253414 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b463259-d990-42c5-8dee-0eacf031fc42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:01:51.254121 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b463259-d990-42c5-8dee-0eacf031fc42 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:51.254403 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b463259-d990-42c5-8dee-0eacf031fc42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:51.254913 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b463259-d990-42c5-8dee-0eacf031fc42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:51.255656 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b463259-d990-42c5-8dee-0eacf031fc42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:01:51.257322 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b463259-d990-42c5-8dee-0eacf031fc42 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:01:51.272708 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1474/2932] 10.222.0.33 () {58 vars in 1121 bytes} [Thu Jun 2 20:01:51 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 02 20:01:51.273885 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d297b1c7-7b93-49a1-9554-ffe202d92bda admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:01:51.290007 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d297b1c7-7b93-49a1-9554-ffe202d92bda admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:01:51.297355 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ebb996-ef35-4927-bc67-23b601f53c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:01:51.298234 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ebb996-ef35-4927-bc67-23b601f53c1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:51.298520 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ebb996-ef35-4927-bc67-23b601f53c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:51.298959 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ebb996-ef35-4927-bc67-23b601f53c1b admin admin] PATH_INFO: `/v3/projects/d37c8d040e474fa381c945cacbfa2446/users/2bdf71883a8c4c449395efb60d4c0ad0/roles/f13fdd0847924014abf1e95594195e78` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:51.300940 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ebb996-ef35-4927-bc67-23b601f53c1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d37c8d040e474fa381c945cacbfa2446, user_id=2bdf71883a8c4c449395efb60d4c0ad0, role_id=f13fdd0847924014abf1e95594195e78)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:01:51.303680 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ebb996-ef35-4927-bc67-23b601f53c1b admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:01:51.315091 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1459/2933] 10.222.0.33 () {60 vars in 1482 bytes} [Thu Jun 2 20:01:51 2022] PUT /identity/v3/projects/d37c8d040e474fa381c945cacbfa2446/users/2bdf71883a8c4c449395efb60d4c0ad0/roles/f13fdd0847924014abf1e95594195e78 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 20:01:57.474464 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c798f-750f-48f5-8891-c3d2d050b5b7 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:57.475417 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c798f-750f-48f5-8891-c3d2d050b5b7 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:57.475759 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c798f-750f-48f5-8891-c3d2d050b5b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:57.505273 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-f73c798f-750f-48f5-8891-c3d2d050b5b7 None None] MFA Rules not processed for user `2bdf71883a8c4c449395efb60d4c0ad0`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 20:01:57.509850 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-f73c798f-750f-48f5-8891-c3d2d050b5b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:01:57.594042 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1475/2934] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 20:01:57 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 20:01:57.608871 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ebb996-ef35-4927-bc67-23b601f53c1b admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:01:57.610790 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-b1ebb996-ef35-4927-bc67-23b601f53c1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:01:57.689760 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ebb996-ef35-4927-bc67-23b601f53c1b admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:01:57.695735 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5087f287-4fb6-4494-be63-d6608c5e9044 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:01:57.696401 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5087f287-4fb6-4494-be63-d6608c5e9044 service nova] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:57.696686 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5087f287-4fb6-4494-be63-d6608c5e9044 service nova] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:57.696962 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5087f287-4fb6-4494-be63-d6608c5e9044 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:57.756674 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5087f287-4fb6-4494-be63-d6608c5e9044 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:01:57.758280 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5087f287-4fb6-4494-be63-d6608c5e9044 service nova] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:01:57.768191 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1460/2935] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 20:01:57 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 20:01:59.527134 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42b4b3-58d2-4226-a14d-126cd5d8b113 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:59.530688 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42b4b3-58d2-4226-a14d-126cd5d8b113 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:59.530688 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42b4b3-58d2-4226-a14d-126cd5d8b113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:59.611134 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-4f42b4b3-58d2-4226-a14d-126cd5d8b113 None None] MFA Rules not processed for user `04f64813f09348eb91cb7bb2869f7612`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 20:01:59.612444 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-4f42b4b3-58d2-4226-a14d-126cd5d8b113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:01:59.640449 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 20:01:59.689359 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1476/2936] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Jun 2 20:01:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 20:01:59.833829 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5087f287-4fb6-4494-be63-d6608c5e9044 service nova] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:01:59.835268 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-5087f287-4fb6-4494-be63-d6608c5e9044 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:01:59.845676 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 20:01:59.929050 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5087f287-4fb6-4494-be63-d6608c5e9044 service nova] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:01:59.934356 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029ceb78-24ef-4652-b32f-e4e2211a0a8d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a59e1d67ff94186a3ebcfcb49938a87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:01:59.935047 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ceb78-24ef-4652-b32f-e4e2211a0a8d service swift] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:01:59.935870 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ceb78-24ef-4652-b32f-e4e2211a0a8d service swift] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:01:59.936308 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ceb78-24ef-4652-b32f-e4e2211a0a8d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:01:59.945211 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029ceb78-24ef-4652-b32f-e4e2211a0a8d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:01:59.946820 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029ceb78-24ef-4652-b32f-e4e2211a0a8d service swift] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:01:59.955594 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1461/2937] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 2 20:01:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 02 20:02:09.785065 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=16 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 20:02:09.971987 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=38 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 20:02:21.469193 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1fd358-eea0-43ca-8125-541d2d0b04dc None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:02:21.469695 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1fd358-eea0-43ca-8125-541d2d0b04dc None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:02:21.469809 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1fd358-eea0-43ca-8125-541d2d0b04dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:02:21.487195 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-0c1fd358-eea0-43ca-8125-541d2d0b04dc None None] MFA Rules not processed for user `04f64813f09348eb91cb7bb2869f7612`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 20:02:21.489067 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-0c1fd358-eea0-43ca-8125-541d2d0b04dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:02:21.495395 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1477/2938] 10.222.0.33 () {58 vars in 967 bytes} [Thu Jun 2 20:02:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 20:02:21.507868 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029ceb78-24ef-4652-b32f-e4e2211a0a8d service swift] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:02:21.519011 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029ceb78-24ef-4652-b32f-e4e2211a0a8d service swift] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:02:21.523957 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e8ee85-5b5d-4877-8d29-7352992facc7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a59e1d67ff94186a3ebcfcb49938a87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:02:21.524593 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e8ee85-5b5d-4877-8d29-7352992facc7 service swift] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:02:21.524880 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e8ee85-5b5d-4877-8d29-7352992facc7 service swift] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:02:21.525449 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e8ee85-5b5d-4877-8d29-7352992facc7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:02:21.533501 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e8ee85-5b5d-4877-8d29-7352992facc7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:02:21.535123 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e8ee85-5b5d-4877-8d29-7352992facc7 service swift] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:02:21.543085 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1462/2939] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Jun 2 20:02:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 02 20:02:22.751709 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0e5680-4502-464f-abf3-bac2ebda1480 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:02:22.751709 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0e5680-4502-464f-abf3-bac2ebda1480 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:02:22.751709 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0e5680-4502-464f-abf3-bac2ebda1480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:02:22.770001 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-5e0e5680-4502-464f-abf3-bac2ebda1480 None None] MFA Rules not processed for user `3a519787e4944a78a93d48bd9e729282`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 20:02:22.771304 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-5e0e5680-4502-464f-abf3-bac2ebda1480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:02:22.778175 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1478/2940] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 20:02:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 02 20:02:24.529986 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e8ee85-5b5d-4877-8d29-7352992facc7 service swift] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:02:24.531600 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-b4e8ee85-5b5d-4877-8d29-7352992facc7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:02:24.597946 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e8ee85-5b5d-4877-8d29-7352992facc7 service swift] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:02:24.602490 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c23eb6-ed4d-4d1b-a171-cd7d53339341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:02:24.603397 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c23eb6-ed4d-4d1b-a171-cd7d53339341 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:02:24.603687 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c23eb6-ed4d-4d1b-a171-cd7d53339341 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:02:24.604048 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c23eb6-ed4d-4d1b-a171-cd7d53339341 admin admin] PATH_INFO: `/v3/users/4f291c269d114ed5b9a692ba80a2a7ef` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:02:24.605231 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c23eb6-ed4d-4d1b-a171-cd7d53339341 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f291c269d114ed5b9a692ba80a2a7ef)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:02:24.607187 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c23eb6-ed4d-4d1b-a171-cd7d53339341 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:02:24.817502 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-93c23eb6-ed4d-4d1b-a171-cd7d53339341 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f291c269d114ed5b9a692ba80a2a7ef'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:02:24.824802 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-93c23eb6-ed4d-4d1b-a171-cd7d53339341 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f291c269d114ed5b9a692ba80a2a7ef'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:02:24.825741 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-93c23eb6-ed4d-4d1b-a171-cd7d53339341 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f291c269d114ed5b9a692ba80a2a7ef'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:02:24.883672 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-93c23eb6-ed4d-4d1b-a171-cd7d53339341 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f291c269d114ed5b9a692ba80a2a7ef'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:02:24.884405 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-93c23eb6-ed4d-4d1b-a171-cd7d53339341 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f291c269d114ed5b9a692ba80a2a7ef'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:02:24.902273 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1463/2941] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 20:02:24 2022] DELETE /identity/v3/users/4f291c269d114ed5b9a692ba80a2a7ef => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 20:02:25.154808 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0e5680-4502-464f-abf3-bac2ebda1480 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:02:25.166787 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0e5680-4502-464f-abf3-bac2ebda1480 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:02:25.229860 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf8f2e6-f357-4a68-bb9c-ad3b5e9db44f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:02:25.231325 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf8f2e6-f357-4a68-bb9c-ad3b5e9db44f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:02:25.231607 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf8f2e6-f357-4a68-bb9c-ad3b5e9db44f admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:02:25.231878 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf8f2e6-f357-4a68-bb9c-ad3b5e9db44f admin admin] PATH_INFO: `/v3/projects/ec39efbb5b894ee7a22d72b70d1f5f7d` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:02:25.233063 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf8f2e6-f357-4a68-bb9c-ad3b5e9db44f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec39efbb5b894ee7a22d72b70d1f5f7d)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:02:25.235000 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf8f2e6-f357-4a68-bb9c-ad3b5e9db44f admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:02:25.349011 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-bcf8f2e6-f357-4a68-bb9c-ad3b5e9db44f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec39efbb5b894ee7a22d72b70d1f5f7d', 'request_id': 'req-bcf8f2e6-f357-4a68-bb9c-ad3b5e9db44f'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:02:25.506119 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-bcf8f2e6-f357-4a68-bb9c-ad3b5e9db44f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec39efbb5b894ee7a22d72b70d1f5f7d', 'request_id': 'req-bcf8f2e6-f357-4a68-bb9c-ad3b5e9db44f'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:02:25.512700 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-bcf8f2e6-f357-4a68-bb9c-ad3b5e9db44f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec39efbb5b894ee7a22d72b70d1f5f7d', 'request_id': 'req-bcf8f2e6-f357-4a68-bb9c-ad3b5e9db44f'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:02:25.523770 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-bcf8f2e6-f357-4a68-bb9c-ad3b5e9db44f admin admin] The token cache is being invalidate because project ec39efbb5b894ee7a22d72b70d1f5f7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 20:02:25.524064 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-bcf8f2e6-f357-4a68-bb9c-ad3b5e9db44f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:02:25.524702 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-bcf8f2e6-f357-4a68-bb9c-ad3b5e9db44f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:02:25.526252 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1479/2942] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 20:02:25 2022] DELETE /identity/v3/projects/ec39efbb5b894ee7a22d72b70d1f5f7d => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 20:02:34.898644 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=59,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 20:02:35.520695 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 20:02:39.423065 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c23eb6-ed4d-4d1b-a171-cd7d53339341 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:02:39.424497 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-93c23eb6-ed4d-4d1b-a171-cd7d53339341 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:02:39.479430 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c23eb6-ed4d-4d1b-a171-cd7d53339341 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:02:39.525071 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4789a4-4c62-4647-84c6-db13be57c1fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:02:39.525713 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4789a4-4c62-4647-84c6-db13be57c1fd service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:02:39.526000 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4789a4-4c62-4647-84c6-db13be57c1fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:02:39.526278 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4789a4-4c62-4647-84c6-db13be57c1fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:02:39.528052 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-bb4789a4-4c62-4647-84c6-db13be57c1fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:02:39.588727 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4789a4-4c62-4647-84c6-db13be57c1fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:02:39.590194 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4789a4-4c62-4647-84c6-db13be57c1fd service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:02:39.600192 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1464/2943] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 20:02:39 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 20:02:49.595903 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=70 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 20:03:22.389724 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf8f2e6-f357-4a68-bb9c-ad3b5e9db44f admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:03:22.391733 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-bcf8f2e6-f357-4a68-bb9c-ad3b5e9db44f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:03:22.403806 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf8f2e6-f357-4a68-bb9c-ad3b5e9db44f admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:03:22.407943 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a493ec-09e8-46f9-b73a-ef79e69f334f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:03:22.408583 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a493ec-09e8-46f9-b73a-ef79e69f334f service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:03:22.408899 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a493ec-09e8-46f9-b73a-ef79e69f334f service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:03:22.409178 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a493ec-09e8-46f9-b73a-ef79e69f334f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:03:22.410935 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-53a493ec-09e8-46f9-b73a-ef79e69f334f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:03:22.420078 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a493ec-09e8-46f9-b73a-ef79e69f334f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:03:22.421512 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a493ec-09e8-46f9-b73a-ef79e69f334f service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:03:22.429428 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1480/2944] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 20:03:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 20:03:32.425895 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=6 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 20:04:34.199689 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4789a4-4c62-4647-84c6-db13be57c1fd service neutron] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:04:34.209557 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4789a4-4c62-4647-84c6-db13be57c1fd service neutron] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:04:34.213201 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab7a4a6-e57f-4dc7-97bc-6b9fd333549a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:04:34.213841 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab7a4a6-e57f-4dc7-97bc-6b9fd333549a service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:04:34.214127 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab7a4a6-e57f-4dc7-97bc-6b9fd333549a service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:04:34.214406 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab7a4a6-e57f-4dc7-97bc-6b9fd333549a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:04:34.216373 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-4ab7a4a6-e57f-4dc7-97bc-6b9fd333549a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:04:34.287096 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab7a4a6-e57f-4dc7-97bc-6b9fd333549a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:04:34.288524 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab7a4a6-e57f-4dc7-97bc-6b9fd333549a service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:04:34.299578 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1465/2945] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 20:04:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 20:04:37.752557 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4cc148-5aa5-492e-b334-213e53490e82 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:04:37.753092 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4cc148-5aa5-492e-b334-213e53490e82 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:04:37.753092 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4cc148-5aa5-492e-b334-213e53490e82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:04:37.771963 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-2d4cc148-5aa5-492e-b334-213e53490e82 None None] MFA Rules not processed for user `3a519787e4944a78a93d48bd9e729282`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 20:04:37.773256 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-2d4cc148-5aa5-492e-b334-213e53490e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:04:37.828444 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1481/2946] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 20:04:37 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 20:04:44.294620 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=34 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 20:04:47.821611 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=25 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 20:05:18.615300 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab7a4a6-e57f-4dc7-97bc-6b9fd333549a service neutron] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:05:18.616746 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-4ab7a4a6-e57f-4dc7-97bc-6b9fd333549a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:05:18.755483 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab7a4a6-e57f-4dc7-97bc-6b9fd333549a service neutron] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:05:18.760636 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de21e1c4-4b0c-4b77-9976-a6bff8f0483d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73ae837e99764bebbe7a6e06ac9049e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:05:18.761284 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21e1c4-4b0c-4b77-9976-a6bff8f0483d service placement] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:05:18.761570 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21e1c4-4b0c-4b77-9976-a6bff8f0483d service placement] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:05:18.761849 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21e1c4-4b0c-4b77-9976-a6bff8f0483d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:05:18.763628 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-de21e1c4-4b0c-4b77-9976-a6bff8f0483d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:05:18.775405 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de21e1c4-4b0c-4b77-9976-a6bff8f0483d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:05:18.776978 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de21e1c4-4b0c-4b77-9976-a6bff8f0483d service placement] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:05:18.787378 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1466/2947] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 20:05:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 20:05:28.783158 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=63 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 20:06:38.576013 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a34581-2e12-4209-88d9-38efb9816164 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:06:38.576532 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a34581-2e12-4209-88d9-38efb9816164 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:06:38.576640 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a34581-2e12-4209-88d9-38efb9816164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:06:38.594493 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-07a34581-2e12-4209-88d9-38efb9816164 None None] MFA Rules not processed for user `3a519787e4944a78a93d48bd9e729282`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 02 20:06:38.595803 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-07a34581-2e12-4209-88d9-38efb9816164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:06:38.602509 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1482/2948] 10.222.0.33 () {58 vars in 957 bytes} [Thu Jun 2 20:06:38 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 02 20:06:38.913432 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de21e1c4-4b0c-4b77-9976-a6bff8f0483d service placement] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:06:38.931643 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de21e1c4-4b0c-4b77-9976-a6bff8f0483d service placement] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:06:38.935164 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bba4624-21ce-42a5-a80a-e5ee847bd0a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:06:38.935788 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba4624-21ce-42a5-a80a-e5ee847bd0a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:06:38.936069 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba4624-21ce-42a5-a80a-e5ee847bd0a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:06:38.936342 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba4624-21ce-42a5-a80a-e5ee847bd0a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:06:38.938079 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-5bba4624-21ce-42a5-a80a-e5ee847bd0a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:06:38.947528 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bba4624-21ce-42a5-a80a-e5ee847bd0a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:06:38.949427 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bba4624-21ce-42a5-a80a-e5ee847bd0a4 service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:06:38.957126 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1467/2949] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 20:06:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 20:06:39.632090 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a34581-2e12-4209-88d9-38efb9816164 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:06:39.642180 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a34581-2e12-4209-88d9-38efb9816164 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:06:39.645819 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8683a0-18e5-444f-ad4e-f697d1351c7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:06:39.646459 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8683a0-18e5-444f-ad4e-f697d1351c7d service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:06:39.646771 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8683a0-18e5-444f-ad4e-f697d1351c7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:06:39.647053 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8683a0-18e5-444f-ad4e-f697d1351c7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:06:39.648846 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-2a8683a0-18e5-444f-ad4e-f697d1351c7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:06:39.684546 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8683a0-18e5-444f-ad4e-f697d1351c7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:06:39.686039 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8683a0-18e5-444f-ad4e-f697d1351c7d service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:06:39.706995 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1483/2950] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 20:06:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 20:06:40.631278 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bba4624-21ce-42a5-a80a-e5ee847bd0a4 service neutron] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:06:40.643412 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bba4624-21ce-42a5-a80a-e5ee847bd0a4 service neutron] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:06:40.648374 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e14be97-2e1e-48f0-a52d-9a6bb07741b0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73ae837e99764bebbe7a6e06ac9049e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:06:40.649048 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14be97-2e1e-48f0-a52d-9a6bb07741b0 service placement] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:06:40.649335 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14be97-2e1e-48f0-a52d-9a6bb07741b0 service placement] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:06:40.649746 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14be97-2e1e-48f0-a52d-9a6bb07741b0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:06:40.651661 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-6e14be97-2e1e-48f0-a52d-9a6bb07741b0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:06:40.662907 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e14be97-2e1e-48f0-a52d-9a6bb07741b0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:06:40.664475 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e14be97-2e1e-48f0-a52d-9a6bb07741b0 service placement] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:06:40.675475 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1468/2951] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 20:06:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 20:06:45.140140 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8683a0-18e5-444f-ad4e-f697d1351c7d service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:06:45.142113 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-2a8683a0-18e5-444f-ad4e-f697d1351c7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:06:45.159747 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8683a0-18e5-444f-ad4e-f697d1351c7d service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:06:45.168150 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb88789a-608c-40c0-86ee-56de4b7d90e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:06:45.173062 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb88789a-608c-40c0-86ee-56de4b7d90e5 service nova] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:06:45.179188 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb88789a-608c-40c0-86ee-56de4b7d90e5 service nova] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:06:45.179188 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb88789a-608c-40c0-86ee-56de4b7d90e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:06:45.179188 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-bb88789a-608c-40c0-86ee-56de4b7d90e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:06:45.189518 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb88789a-608c-40c0-86ee-56de4b7d90e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:06:45.191241 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb88789a-608c-40c0-86ee-56de4b7d90e5 service nova] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:06:45.209497 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1484/2952] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 20:06:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 20:06:50.670925 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=16 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 20:06:55.198304 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=27 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 20:06:56.482195 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e14be97-2e1e-48f0-a52d-9a6bb07741b0 service placement] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:06:56.483928 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-6e14be97-2e1e-48f0-a52d-9a6bb07741b0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:06:56.498994 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e14be97-2e1e-48f0-a52d-9a6bb07741b0 service placement] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:06:56.504395 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0059bb79-f6c3-4a6c-b3fa-e98fe442c5b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:06:56.505028 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0059bb79-f6c3-4a6c-b3fa-e98fe442c5b2 service nova] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:06:56.505315 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0059bb79-f6c3-4a6c-b3fa-e98fe442c5b2 service nova] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:06:56.505588 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0059bb79-f6c3-4a6c-b3fa-e98fe442c5b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:06:56.507346 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-0059bb79-f6c3-4a6c-b3fa-e98fe442c5b2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:06:56.519816 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0059bb79-f6c3-4a6c-b3fa-e98fe442c5b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:06:56.521359 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0059bb79-f6c3-4a6c-b3fa-e98fe442c5b2 service nova] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:06:56.531696 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1469/2953] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 20:06:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 20:06:57.000064 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb88789a-608c-40c0-86ee-56de4b7d90e5 service nova] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:06:57.011086 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb88789a-608c-40c0-86ee-56de4b7d90e5 service nova] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:06:57.016504 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4bc463-7173-4215-bbea-18933daa6f4a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:06:57.017140 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4bc463-7173-4215-bbea-18933daa6f4a service nova] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:06:57.017422 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4bc463-7173-4215-bbea-18933daa6f4a service nova] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:06:57.017696 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4bc463-7173-4215-bbea-18933daa6f4a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:06:57.019459 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-5d4bc463-7173-4215-bbea-18933daa6f4a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:06:57.029365 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4bc463-7173-4215-bbea-18933daa6f4a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:06:57.030969 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4bc463-7173-4215-bbea-18933daa6f4a service nova] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:06:57.039647 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1485/2954] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 20:06:56 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 20:07:06.527258 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=6 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 20:07:07.035739 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=6 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 20:08:28.347965 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0059bb79-f6c3-4a6c-b3fa-e98fe442c5b2 service nova] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:08:28.357937 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0059bb79-f6c3-4a6c-b3fa-e98fe442c5b2 service nova] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:08:28.361514 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006696b5-2ce9-4720-9ffb-e8cb4045ab92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:08:28.362142 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006696b5-2ce9-4720-9ffb-e8cb4045ab92 service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:08:28.362428 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006696b5-2ce9-4720-9ffb-e8cb4045ab92 service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:08:28.362706 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006696b5-2ce9-4720-9ffb-e8cb4045ab92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:08:28.369941 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006696b5-2ce9-4720-9ffb-e8cb4045ab92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:08:28.371353 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006696b5-2ce9-4720-9ffb-e8cb4045ab92 service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:08:28.379274 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1470/2955] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 20:08:28 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 20:08:29.893213 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4bc463-7173-4215-bbea-18933daa6f4a service nova] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:08:29.906607 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4bc463-7173-4215-bbea-18933daa6f4a service nova] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:08:29.912975 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e968c94-3c49-49b5-aac0-f394b82ded83 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70664d5cbdee47ee89c120761939cc75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:08:29.913691 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e968c94-3c49-49b5-aac0-f394b82ded83 service nova] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:08:29.913980 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e968c94-3c49-49b5-aac0-f394b82ded83 service nova] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:08:29.914258 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e968c94-3c49-49b5-aac0-f394b82ded83 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:08:29.916112 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-1e968c94-3c49-49b5-aac0-f394b82ded83 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:08:29.929252 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e968c94-3c49-49b5-aac0-f394b82ded83 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:08:29.931697 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e968c94-3c49-49b5-aac0-f394b82ded83 service nova] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:08:29.944535 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1486/2956] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 20:08:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 20:08:38.375734 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=6 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 20:08:39.939166 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=6 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 20:08:43.121107 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006696b5-2ce9-4720-9ffb-e8cb4045ab92 service neutron] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:08:43.129876 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006696b5-2ce9-4720-9ffb-e8cb4045ab92 service neutron] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:08:43.133459 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e197655d-2e95-41ed-b46f-b88d6468d418 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7417a54046e34e899d7b03f2f2314deb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:08:43.134101 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e197655d-2e95-41ed-b46f-b88d6468d418 service neutron] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:08:43.134386 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e197655d-2e95-41ed-b46f-b88d6468d418 service neutron] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:08:43.134665 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e197655d-2e95-41ed-b46f-b88d6468d418 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:08:43.136473 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-e197655d-2e95-41ed-b46f-b88d6468d418 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:08:43.218550 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e197655d-2e95-41ed-b46f-b88d6468d418 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:08:43.220030 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e197655d-2e95-41ed-b46f-b88d6468d418 service neutron] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:08:43.238848 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1471/2957] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 20:08:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 20:08:46.360531 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e968c94-3c49-49b5-aac0-f394b82ded83 service nova] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:08:46.371703 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e968c94-3c49-49b5-aac0-f394b82ded83 service nova] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:08:46.376659 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3555ad0a-8ac3-4c9d-a20d-f172f5087214 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73ae837e99764bebbe7a6e06ac9049e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:08:46.377294 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3555ad0a-8ac3-4c9d-a20d-f172f5087214 service placement] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:08:46.377577 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3555ad0a-8ac3-4c9d-a20d-f172f5087214 service placement] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:08:46.377849 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3555ad0a-8ac3-4c9d-a20d-f172f5087214 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:08:46.387388 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-3555ad0a-8ac3-4c9d-a20d-f172f5087214 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:08:46.392307 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3555ad0a-8ac3-4c9d-a20d-f172f5087214 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:08:46.393927 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3555ad0a-8ac3-4c9d-a20d-f172f5087214 service placement] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:08:46.405887 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1487/2958] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 20:08:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 20:08:49.981979 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e197655d-2e95-41ed-b46f-b88d6468d418 service neutron] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:08:49.993219 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e197655d-2e95-41ed-b46f-b88d6468d418 service neutron] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:08:49.998132 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26614232-d936-41f6-b5bc-cd6e93446fe4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73ae837e99764bebbe7a6e06ac9049e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:08:50.000780 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26614232-d936-41f6-b5bc-cd6e93446fe4 service placement] REQUEST_METHOD: `GET` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:08:50.000780 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26614232-d936-41f6-b5bc-cd6e93446fe4 service placement] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:08:50.000780 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26614232-d936-41f6-b5bc-cd6e93446fe4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:08:50.001113 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-26614232-d936-41f6-b5bc-cd6e93446fe4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:08:50.012550 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26614232-d936-41f6-b5bc-cd6e93446fe4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:08:50.014107 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26614232-d936-41f6-b5bc-cd6e93446fe4 service placement] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:08:50.026093 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1472/2959] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 20:08:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 02 20:08:50.102711 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3555ad0a-8ac3-4c9d-a20d-f172f5087214 service placement] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:08:50.104829 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-3555ad0a-8ac3-4c9d-a20d-f172f5087214 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:08:50.118389 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3555ad0a-8ac3-4c9d-a20d-f172f5087214 service placement] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:08:50.122942 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3638a259-3d0a-49df-bc13-3b34aeacdf8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:08:50.123820 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638a259-3d0a-49df-bc13-3b34aeacdf8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:08:50.124102 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638a259-3d0a-49df-bc13-3b34aeacdf8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:08:50.124376 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638a259-3d0a-49df-bc13-3b34aeacdf8b admin admin] PATH_INFO: `/v3/users/3a519787e4944a78a93d48bd9e729282` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:08:50.125556 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3638a259-3d0a-49df-bc13-3b34aeacdf8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a519787e4944a78a93d48bd9e729282)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:08:50.127517 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3638a259-3d0a-49df-bc13-3b34aeacdf8b admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:08:50.317927 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-3638a259-3d0a-49df-bc13-3b34aeacdf8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a519787e4944a78a93d48bd9e729282'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:08:50.325135 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-3638a259-3d0a-49df-bc13-3b34aeacdf8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a519787e4944a78a93d48bd9e729282'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:08:50.381310 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-3638a259-3d0a-49df-bc13-3b34aeacdf8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a519787e4944a78a93d48bd9e729282'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:08:50.381878 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-3638a259-3d0a-49df-bc13-3b34aeacdf8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a519787e4944a78a93d48bd9e729282'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:08:50.382531 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-3638a259-3d0a-49df-bc13-3b34aeacdf8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a519787e4944a78a93d48bd9e729282'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:08:50.401062 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1488/2960] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 20:08:50 2022] DELETE /identity/v3/users/3a519787e4944a78a93d48bd9e729282 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 20:08:50.638140 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26614232-d936-41f6-b5bc-cd6e93446fe4 service placement] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:08:50.648545 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26614232-d936-41f6-b5bc-cd6e93446fe4 service placement] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:08:50.705268 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62d6824-b0a3-4fcc-b8d3-991065b25c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:08:50.706139 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62d6824-b0a3-4fcc-b8d3-991065b25c03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:08:50.706425 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62d6824-b0a3-4fcc-b8d3-991065b25c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:08:50.706701 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62d6824-b0a3-4fcc-b8d3-991065b25c03 admin admin] PATH_INFO: `/v3/projects/4a81da1e3d824c7691c755d265050732` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:08:50.707927 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62d6824-b0a3-4fcc-b8d3-991065b25c03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a81da1e3d824c7691c755d265050732)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:08:50.709817 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62d6824-b0a3-4fcc-b8d3-991065b25c03 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:08:50.842885 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-e62d6824-b0a3-4fcc-b8d3-991065b25c03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a81da1e3d824c7691c755d265050732', 'request_id': 'req-e62d6824-b0a3-4fcc-b8d3-991065b25c03'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:08:50.846446 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 20:08:50.941036 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-e62d6824-b0a3-4fcc-b8d3-991065b25c03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a81da1e3d824c7691c755d265050732', 'request_id': 'req-e62d6824-b0a3-4fcc-b8d3-991065b25c03'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:08:50.946792 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-e62d6824-b0a3-4fcc-b8d3-991065b25c03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a81da1e3d824c7691c755d265050732', 'request_id': 'req-e62d6824-b0a3-4fcc-b8d3-991065b25c03'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:08:50.958109 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-e62d6824-b0a3-4fcc-b8d3-991065b25c03 admin admin] The token cache is being invalidate because project 4a81da1e3d824c7691c755d265050732 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 20:08:50.959438 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-e62d6824-b0a3-4fcc-b8d3-991065b25c03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:08:50.959438 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-e62d6824-b0a3-4fcc-b8d3-991065b25c03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:08:50.960704 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1473/2961] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 20:08:50 2022] DELETE /identity/v3/projects/4a81da1e3d824c7691c755d265050732 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 02 20:08:50.965286 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3638a259-3d0a-49df-bc13-3b34aeacdf8b admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:08:50.966718 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-3638a259-3d0a-49df-bc13-3b34aeacdf8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:08:50.980212 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3638a259-3d0a-49df-bc13-3b34aeacdf8b admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:08:51.035103 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f67f061-181d-4d34-9e8f-bc115180aafd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:08:51.035972 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f67f061-181d-4d34-9e8f-bc115180aafd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:08:51.036255 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f67f061-181d-4d34-9e8f-bc115180aafd admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:08:51.036527 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f67f061-181d-4d34-9e8f-bc115180aafd admin admin] PATH_INFO: `/v3/users/2bdf71883a8c4c449395efb60d4c0ad0` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:08:51.037687 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f67f061-181d-4d34-9e8f-bc115180aafd admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bdf71883a8c4c449395efb60d4c0ad0)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:08:51.039613 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f67f061-181d-4d34-9e8f-bc115180aafd admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:08:51.249430 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-2f67f061-181d-4d34-9e8f-bc115180aafd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bdf71883a8c4c449395efb60d4c0ad0'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:08:51.256269 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-2f67f061-181d-4d34-9e8f-bc115180aafd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bdf71883a8c4c449395efb60d4c0ad0'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:08:51.260371 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 20:08:51.327380 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-2f67f061-181d-4d34-9e8f-bc115180aafd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bdf71883a8c4c449395efb60d4c0ad0'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:08:51.328057 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-2f67f061-181d-4d34-9e8f-bc115180aafd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bdf71883a8c4c449395efb60d4c0ad0'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:08:51.328665 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-2f67f061-181d-4d34-9e8f-bc115180aafd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bdf71883a8c4c449395efb60d4c0ad0'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:08:51.346442 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1489/2962] 10.222.0.33 () {58 vars in 1223 bytes} [Thu Jun 2 20:08:50 2022] DELETE /identity/v3/users/2bdf71883a8c4c449395efb60d4c0ad0 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 20:08:51.603799 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62d6824-b0a3-4fcc-b8d3-991065b25c03 admin admin] Authenticating user token {{(pid=80492) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:08:51.606360 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.fernet_utils [None req-e62d6824-b0a3-4fcc-b8d3-991065b25c03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80492) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:08:51.621543 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62d6824-b0a3-4fcc-b8d3-991065b25c03 admin admin] Validating token access rules against request {{(pid=80492) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:08:51.678727 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db860a01-6b3d-4f69-96d6-750b7f6de643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789d0dc0871243f79a6f4f6f602c4f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '879ee62507da4a8282b2418099da56e5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80492) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:08:51.679612 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db860a01-6b3d-4f69-96d6-750b7f6de643 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:08:51.679895 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db860a01-6b3d-4f69-96d6-750b7f6de643 admin admin] SCRIPT_NAME: `/identity` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:08:51.680164 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db860a01-6b3d-4f69-96d6-750b7f6de643 admin admin] PATH_INFO: `/v3/projects/d37c8d040e474fa381c945cacbfa2446` {{(pid=80492) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:08:51.681320 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db860a01-6b3d-4f69-96d6-750b7f6de643 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d37c8d040e474fa381c945cacbfa2446)` {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:08:51.683744 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db860a01-6b3d-4f69-96d6-750b7f6de643 admin admin] RBAC: Authorization granted {{(pid=80492) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:08:51.779278 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-db860a01-6b3d-4f69-96d6-750b7f6de643 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd37c8d040e474fa381c945cacbfa2446', 'request_id': 'req-db860a01-6b3d-4f69-96d6-750b7f6de643'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:08:51.859508 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-db860a01-6b3d-4f69-96d6-750b7f6de643 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd37c8d040e474fa381c945cacbfa2446', 'request_id': 'req-db860a01-6b3d-4f69-96d6-750b7f6de643'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:08:51.865461 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-db860a01-6b3d-4f69-96d6-750b7f6de643 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd37c8d040e474fa381c945cacbfa2446', 'request_id': 'req-db860a01-6b3d-4f69-96d6-750b7f6de643'} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:08:51.877281 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-db860a01-6b3d-4f69-96d6-750b7f6de643 admin admin] The token cache is being invalidate because project d37c8d040e474fa381c945cacbfa2446 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80492) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 02 20:08:51.877576 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-db860a01-6b3d-4f69-96d6-750b7f6de643 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:08:51.878241 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG keystone.notifications [None req-db860a01-6b3d-4f69-96d6-750b7f6de643 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80492) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 02 20:08:51.879846 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: [pid: 80492|app: 0|req: 1474/2963] 10.222.0.33 () {58 vars in 1232 bytes} [Thu Jun 2 20:08:51 2022] DELETE /identity/v3/projects/d37c8d040e474fa381c945cacbfa2446 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 02 20:09:01.453723 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=8,keystone:DELETE=2 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 20:09:01.874249 dv-cinder-830901-8-iscsi devstack@keystone.service[80492]: DEBUG dbcounter [-] [80492] Writing DB stats keystone:SELECT=57,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80492) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 02 20:10:24.584195 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f67f061-181d-4d34-9e8f-bc115180aafd admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 02 20:10:24.585638 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-2f67f061-181d-4d34-9e8f-bc115180aafd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:10:24.654061 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f67f061-181d-4d34-9e8f-bc115180aafd admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 02 20:10:24.846819 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e598c799-f8a5-4e44-a393-e2b792a048d1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73ae837e99764bebbe7a6e06ac9049e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c04020eaebd4647804cf8653f32d9e5', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 02 20:10:24.847370 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598c799-f8a5-4e44-a393-e2b792a048d1 service placement] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 02 20:10:24.847654 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598c799-f8a5-4e44-a393-e2b792a048d1 service placement] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 02 20:10:24.847925 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598c799-f8a5-4e44-a393-e2b792a048d1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 02 20:10:24.849669 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-e598c799-f8a5-4e44-a393-e2b792a048d1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 02 20:10:24.861222 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e598c799-f8a5-4e44-a393-e2b792a048d1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 02 20:10:24.862800 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e598c799-f8a5-4e44-a393-e2b792a048d1 service placement] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 02 20:10:24.873025 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1490/2964] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Jun 2 20:10:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 02 20:10:34.868872 dv-cinder-830901-8-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=58 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}